Create an SFTP Data Source

Configure an SFTP Data Source to extract data from an external SFTP Location.

What is an SFTP Data Source?

We use SFTP Data Sources to connect to any SFTP server hosted outside One Model that stores your data. This is separate to SFTP Users for a File Data Source, which allow users to upload data to a One Model hosted SFTP Server.

 

Create an SFTP Data Source

To Create an SFTP Data Source, go to the Data Sources page and select SFTP from the Add Data Sources menu:

 

From there you'll be able to configure these options:

  • Name: The name of the SFTP Data Source

  • Data Loads Should Process Data: enable if you want Model Processing to run once Data has loaded

  • Host: The Host URL of the SFTP Server being connected to

  • Port: The Port number of the SFTP (usually 22)

  • Username: The username configured on the SFTP Server

  • Password: The password configured for the user on the SFTP Server

Add Files to your SFTP Data Source

Once you've created your SFTP Data Source, you can add some files. There are four kinds of files that can be added:

  • Delimited Input: This is for flat files

  • JSON Input: This is for files written in JSON, which may contain multiple objects/tables

  • XML Input: This is for files written in XML, which may also contain multiple objects/tables

  • Compressed File input: This is for cases where a single Compressed archive (like a Zip File) contains more than one file of the above three types

Configuration Options available for all Input types:

In all Input types, the following configuration options are possible:

  • SFTPFilePath: This is the path for the folder on the SFTP Server that the desired files are located. Use “/<myFolderName>/” for files contained in folders, or “/” for files not contained in folders.

  • SFTPFileNamePattern: This is the expected name for the pattern. A % can be used as a wildcard to denote part of the name that changes (such as a timestamp). This is only necessary for the ExactFileNameMatch Match Method

  • SFTPMatchMethod: This is the method used to identify files that need to be pulled when the SFTP Data Source is run.

  • SFTPNewSinceUtc: ??

Add Compressed Folder Input

Note: This is only necessary for cases where a single compressed file contains multiple files of another type. While this is available for the SFTP Data Source, it is currently not available for the File Data Source

 

To add a Compressed Folder Input, simply configure the options specified above to find the file name. The only specific configuration option for the Compressed Folder input is:

  • addCompressedFolderNameToFiles: True/False. If set to True, this adds the Compressed Folder name to all files contained within the file (before the extension)

Add Delimited File Input

Further details for how to use Delimited File Inputs can be found here.

If you have questions about SFTP, be sure to check out our SFTP FAQs here

Was this article helpful?

0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.