5.1 Teradata Parallel Transporter - Data Connector Operator

DataConnector capabilities in brief:

Data Connector Operator can play Dual role :

• It can act as Producer Operator to read data from file or from access module.

• It can also act as Consumer Operator to write data to flat files or access module.

Data Connector can also used for scanning directories.

We can use multiple instances of the operator.

DataConnector Operator as Producer:

Data connector can read files in multiple ways as floows:

1. Read a single flat file by specifying a directory path and file name.

2. Read a directory of flat files by specifying a directory path and a wildcard character (*) with the file name. The wildcard character allows a directory that contains hundreds of files to be treated as a single data source.

3. Read data using the supported access modules

Following is the example of the DataConnector Producer:

DEFINE DATACONNECTOR OPERATOR

DESCRIPTION 'DataConnector'

TYPE DATACONNECTOR PRODUCER --> This is mandatory to make the operator as Data Connector producer.

SCHEMA PRODUCT_SOURCE_SCHEMA

ATTRIBUTES

(

VARCHAR AccessModuleName,

VARCHAR AccessModuleInitStr,

VARCHAR FileName,

VARCHAR Format,

VARCHAR OpenMode,

VARCHAR TextDelimiter,

VARCHAR ArchiveDirectoryPath,

VARCHAR DirectoryPath,

VARCHAR PrivateLogName,

VARCHAR VigilElapsedTime,

VARCHAR VigilWaitTime,

VARCHAR VigilStartTime,

VARCHAR VigilStopTime,

VARCHAR VigilSortField

);

Depending upon the Way we are using Data Connector Producer the key attributes differ.

Operator Usage	Key Attributes
Data Connector Operator  as producer without Access Module	• FileList • FileName • Format • MultipleReaders • OpenMode • Te x t D e limit e r • PrivateLogName
Data Connector Operator  as producer with Access Module	• AccessModuleName • AccessModuleInitStr • FileName • Format • OpenMode • Te x t D e limit e r • PrivateLogName
Data Connector Operator as producer using Batch Scan. Batch Scan: The job runs on a specific time or is manually run and it reads all the files in a given directory that satisfy the filename we give.	• ArchiveDirectoryPath • DirectoryPath • EnableScan • FileName • Format • OpenMode • Te x t D e limit e r • PrivateLogName
Data Connector Operator as Producer using Active Scan. In Active scan the job keeps looking for the file in the given directory and starts processing as soon as the file comes.	• ArchiveDirectoryPath • DirectoryPath • EnableScan • FileName • Format • OpenMode • Te x t D e limit e r • VigilElapsedTime • VigilStartTime • VigilSortFile • VigilWaitTime • PrivateLogName

DataConnector Operator as Consumer:

Following is the example of Data Connector as Consumer:

DEFINE DATACONNECTOR OPERATOR

DESCRIPTION 'DataConnector'

TYPE DATACONNECTOR CONSUMER --> Note this should be the TYPE to make it a Data Connector Consumer operator

SCHEMA CONSUMER_SOURCE_SCHEMA

ATTRIBUTES

(

VARCHAR AccessModuleName,

VARCHAR AccessModuleInitStr,

VARCHAR FileName,

VARCHAR Format,

VARCHAR OpenMode,

VARCHAR Text Delimiter,

VARCHAR PrivateLogName

);

Depending upon the Way we are using Data Connector Consumer the key attributes differ.

Operator Usage	Key Attributes
Data Connector Operator  as producer without Access Module	• FileList • FileName • Format • OpenMode • Te x t D e limit e r • PrivateLogName
Data Connector Operator  as producer with Access Module	• AccessModuleName • AccessModuleInitStr • FileName • Format • OpenMode • Te x t D e limit e r • PrivateLogName