54 Questions
What does the 'Stop On Condition' property do in Matillion ETL?
Stops the iteration based on a specified condition
What method is used to create conditions in the Simple mode of Matillion ETL?
Selecting from drop-down menus and text fields
What is the role of the 'Qualifier' in the Simple mode condition creation in Matillion ETL?
Comparing the input variable to the value
In which mode of condition creation must users manually write the condition in JavaScript in Matillion ETL?
Advanced mode
What does the 'Combine Conditions' property allow users to do in Matillion ETL?
Combine defined conditions using 'And' or 'Or'
Which property makes values like 'Iteration Attempted' available for export in Matillion ETL?
Variable Exports
'Not' in the Qualifier property does what to comparisons in Matillion ETL?
'Equals' becomes 'Not equals', 'Less than' becomes 'Greater than or equal to', etc.
'Iterators terminate after failure' refers to what in Matillion ETL?
'Iteration Successful' is equal to max iteration number minus failures
'Source Description Iteration Attempted' refers to what value in Matillion ETL?
'Iteration Attempted'
Which statement accurately describes the File Iterator component?
It enables users to loop over matching files in a remote file system.
How does the File Iterator component handle the execution of attached components?
It runs the attached component once for each file found.
How are file names and path names mapped in the File Iterator component?
They are mapped into environment variables, which can then be referenced from the attached component(s).
What is the recommended approach for iterating more than one component using the File Iterator?
Create a separate orchestration job or transformation job and use a Run Transformation or Run Orchestration component attached to the iterator.
What is the maximum number of iterations allowed for all iterator components?
5000 iterations
What is the purpose of the 'Name' property in the File Iterator component?
It is a human-readable name for the component.
Which remote file system types are available for selection in the 'Input Data Type' property?
Azure Blob Storage, Cloud Storage, FTP, HDFS, S3, SFTP, and Windows Fileshare
What is the purpose of the 'Input Data URL' property?
It provides the URL, including the full path and file name followed by a forward slash, that will point to the files to download to the selected staging area.
What is the purpose of the Filter Regex property?
To define the folder structure and file pattern to search for
What is the significance of the ${jv_folder}/.* expression in the Filter Regex?
It specifies to search for all files within a specific folder and its subfolders
What is the purpose of the Recursive property?
To determine whether to search for files within subdirectories
What is the purpose of the Variables property?
To define environment variables and file attributes to export for each matched file
What is the purpose of the Break on Failure property?
To control whether to stop or continue iterations if a failure occurs
What is the difference between the Concurrent and Sequential settings for the Concurrency property?
Concurrent runs iterations in parallel, while Sequential runs them one after the other
What is the purpose of the Set Home Directory as Root property?
To define whether the URL path is relative to the server root or the user's home directory
What is the purpose of the Max Recursion Depth property?
To specify the maximum depth to search for files in subdirectories
What action will be taken if the 'Break on Failure' property is set to 'Yes'?
Fail immediately if the attached component doesn't run successfully.
Which property in the File Iterator component allows users to specify whether to search for files within subdirectories or not?
Recursive
What does the 'Concurrency' property in the File Iterator component determine?
How iterations run (concurrently or sequentially)
When is the 'Ignore Hidden' property used in the File Iterator component?
To ignore 'hidden' files even if they match the filter regex
What does the 'Filter Regex' property in the File Iterator component define?
The Java-standard regular expression used to test against file paths
What does setting 'Max Recursion Depth' property to a value restrict in the File Iterator component?
The maximum recursion depth into subdirectories
What is the significance of setting 'Set Home Directory as Root' to 'Yes'?
'Hidden' files are included.
'Concurrent: Iterations run concurrently.' is associated with which property in the File Iterator component?
'Concurrency'
'Sequential: Iterations run in sequence, waiting for each to complete before starting the next.' refers to which property in the File Iterator component?
'Concurrency'
In Matillion ETL, what is the purpose of the 'Stop On Condition' property?
To terminate the iteration based on a specified condition.
What does the 'Combine Conditions' property enable users to do in Matillion ETL?
Combine multiple conditions with logical operators.
If a user selects the 'Advanced' mode for condition creation in Matillion ETL, what is the method of specifying conditions?
By manually writing conditions using JavaScript.
What does the 'Qualifier' property in Simple mode condition creation allow users to do?
Reverse the effect of comparisons.
What does the 'Iterators terminate after failure' statement refer to in Matillion ETL?
The number of successful iterations minus failures.
When specifying conditions using Simple mode in Matillion ETL, what does selecting 'Not' as a Qualifier do?
Reverses the effect of the comparison operators.
'Source Description Iteration Attempted' in Matillion ETL specifically refers to what value?
Number of iterations that have been initiated.
'Combine Conditions' property is only available in Matillion ETL when which Mode is set?
'Simple'
'Variable Exports' property in Matillion ETL allows for exporting which values into variables?
'Iteration Attempted' and 'Iteration Successful'
Which property allows users to set the method of creating conditions as either 'Simple' or 'Advanced' in Matillion ETL?
'Mode'
How does the File Iterator component handle the execution of attached components?
It runs the attached component once for each file found in the specified remote file system.
How are file names and path names mapped in the File Iterator component?
They are mapped into environment variables that can be referenced from the attached component(s).
What is the recommended approach for iterating more than one component using the File Iterator?
Put the components into a separate orchestration job or transformation job and use a Run Transformation or Run Orchestration component attached to the iterator.
What is the purpose of the Input Data URL property?
It defines the URL, including the full path and file name followed by a forward slash, that will point to the files to download to the selected staging area.
What is the maximum number of iterations allowed for all iterator components?
5000 iterations
Which remote file system types are available for selection in the Input Data Type property?
Azure Blob Storage, Cloud Storage, FTP, HDFS, S3, SFTP, and Windows Fileshare
What is the purpose of the Name property in the File Iterator component?
It sets a human-readable name for the component.
What is the purpose of the Recursive property?
It determines whether the File Iterator should search for files in nested directories or only in the top-level directory specified by the Input Data URL.
What is the significance of the ${jv_folder}/.* expression in the Filter Regex property?
It instructs the File Iterator to search for all files in the specified directory and its subdirectories.
How does the Variables property impact the execution of the File Iterator component?
It allows users to define custom environment variables that can be referenced from the attached component(s).
Study Notes
File Iterator Component
- The File Iterator component allows users to loop over matching files in a remote file system.
- It searches for files in various remote file systems, running its attached component once for each file found.
- Filenames and path names are mapped into environment variables, which can then be referenced from the attached component(s).
Component Properties
- Name: A human-readable name for the component.
- Input Data Type: Select the remote file system to search, including Azure Blob Storage, Cloud Storage, FTP, HDFS, S3, SFTP, and Windows Fileshare.
- Input Data URL: Input the URL, including the full path and file name followed by a forward slash, that will point to the files to download to the selected staging area.
Iteration Properties
- Record Values In Task History: Choose whether to record iteration values in the Matillion ETL Task History. Default setting is Yes.
- Stop On Condition: Select Yes to stop the iteration based on a condition specified in the Condition property. Default setting is No.
- Mode: Select the method of creating the condition, either Simple or Advanced.
- Condition (Simple mode): Specify an Input Variable, Qualifier, Comparator, and Value using drop-down menus and text fields.
- Condition (Advanced mode): Manually write the condition in the editor using JavaScript.
- Combine Conditions: Use the defined conditions in combination with one another according to either And or Or.
Variable Exports
- Iteration Attempted: The number of iterations that this component attempts to reach (Max Iterations parameter).
- Iteration Generated: The number of iterations that have been initiated.
- Iteration Successful: The number of iterations successfully performed.
Concurrency Properties
- Concurrency: Select whether iterations run concurrently or sequentially. Default setting is Sequential.
- Break on Failure: Select whether to attempt to run the attached component for each iteration, regardless of success or failure. Default setting is No.
File Attributes
- Variables: An existing environment variable to hold the given value of the Path Selection.
- File Attribute: For each matched file, the target variable can be populated with the Base Path, the Subfolder, the Filename, or the date of when the file was Last Modified.
Learn the functionality of the File Iterator component which enables users to loop over matching files in a remote file system. Discover how filenames and path names are mapped into environment variables for easy referencing in attached components. Find out how to attach and detach the iterator to other components.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
