Configure an SFTP Data Source to extract data from an external SFTP Location.
What is an SFTP Data Source?
We use SFTP Data Sources to connect to any SFTP server hosted outside One Model that stores your data. This is separate to SFTP Users for a File Data Source, which allow users to upload data to a One Model hosted SFTP Server.
Create an SFTP Data Source
To Create an SFTP Data Source, go to the Data Sources page and select SFTP from the Add Data Sources menu:
From there you'll be able to configure these options:
-
Name: The name of the SFTP Data Source
-
Data Loads Should Process Data: enable if you want Model Processing to run once Data has loaded
-
Host: The Host URL of the SFTP Server being connected to
-
Port: The Port number of the SFTP (usually 22)
-
Username: The username configured on the SFTP Server
-
Password: The password configured for the user on the SFTP Server
Add Files to your SFTP Data Source
Once you've created your SFTP Data Source, you can add some files. There are four kinds of files that can be added:
-
Delimited Input: This is for flat files
-
JSON Input: This is for files written in JSON, which may contain multiple objects/tables
-
XML Input: This is for files written in XML, which may also contain multiple objects/tables
-
Compressed File input: This is for cases where a single Compressed archive (like a Zip File) contains more than one file of the above three types
Configuration Options available for all Input types:
In all Input types, the following configuration options are possible:
-
SFTPFilePath: This is the path for the folder on the SFTP Server that the desired files are located. Use “/<myFolderName>/” for files contained in folders, or “/” for files not contained in folders.
-
SFTPFileNamePattern: This is the expected name for the pattern. A % can be used as a wildcard to denote part of the name that changes (such as a timestamp). This is only necessary for the ExactFileNameMatch Match Method
-
SFTPMatchMethod: This is the method used to identify files that need to be pulled when the SFTP Data Source is run.
Add Compressed Folder Input
Note: This is only necessary for cases where a single compressed file contains multiple files of another type. While this is available for the SFTP Data Source, it is currently not available for the File Data Source
To add a Compressed Folder Input, simply configure the options specified above to find the file name. The only specific configuration option for the Compressed Folder input is:
-
addCompressedFolderNameToFiles: True/False. If set to True, this adds the Compressed Folder name to all files contained within the file (before the extension)
Add Delimited File Input
Further details for how to use Delimited File Inputs can be found here.
If you have questions about SFTP, be sure to check out our SFTP FAQs here.
Comments
0 comments
Please sign in to leave a comment.