Contents

1 Overview 2

1.1 Workflow 2

2 Configuration 3

2.1 Managing Preprocessors 3

2.2 Configure a Preprocessor 4

2.2.1 Preprocessor configuration values 4

2.3 Example configuration 5

2.3.1 Preprocessor file path 5

2.3.2 Parameters 6

Overview

A preprocessor is a computer program that modifies data to conform with the input requirements of another program. The Import Manager provides a means to automatically run a program or script that will preprocess a copy of the import file prior to importing that copy of the file into PypeServer. Any program or script can be used, allowing automated replacement of data with new values, adding additional data, and cross-referencing data with other files.

Workflow

When an importer pre-processor is selected, the Import Manager will create a copy of the original file and pass that file to the selected pre-processor for processing. When the preprocessor is no longer running and has exited, the Importer Manager will pass the pre-processed file to the importer where the import processing will occur.

Configuration

Preprocessors can be configured and selected from the Preprocessor tab of the Import Manager

Managing Preprocessors

A preprocessor configuration consists of a command line, one or more optional parameters, and a choice of deleting or keeping the preprocessed file copies after importing. Each preprocessor configuration can be created, saved, renamed, deleted, exported and imported.

A set of buttons at the top of the Preprocessor tab handle the creation and management of all of an importer's preprocessors.

Preprocessor toolbar area

Preprocessor buttons - from left to right

• Save - Save the current preprocessor

• Create - Create a new preprocessor

• Copy - Copy the current preprocessor to a new one

• Rename - Rename the current preprocessor

• Delete - Delete the current preprocessor

• Import - Import a preprocessor

• Export - Export the selected preprocessor

• Help - Displays help documentation

Preprocessor dropdown list

• Selects a preprocessor from a list of saved preprocessors for this importer

Configure a Preprocessor

Once a new preprocessor has been created using the Create button, it can be configured using the remaining controls below preprocessor dropdown list.

Preprocessor configuration values

Preprocessor file path - required

The filename of the preprocessor executable or script, without any parameters. If a full path is specified it will be used. If just the filename is specified, then the operating system will use the paths environment variable to locate the file.

Examples

• powershell.exe

• c:\windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe

Parameters - required

Contains one or more parameters that will be passed on the command line after the preprocessor file path. These parameters must always include the mandatory parameter keyword "{filename}", as this keyword is replaced during preprocessing with the import file name.

Examples

Pass just the import filename to the preprocessor on the command line

• {filename}

Pass a list of parameters which include the filename token

• -ExecutionPolicy UnRestricted -File "\\pypeserver\PypeServer Documents\PreProcessors\General\PCFReplacePreProcessor.ps1" "\\pypeserver\PypeServer Documents\PreProcessors\General\PCFLookup.csv" {filename}

Keep preprocessed files after import

The preprocessed copy of the import file is usually deleted after import. Check this box to keep the preprocessed file from being deleted. These files can be found in the same folder as the original import files. These files will have the same names as the originals with the suffix "_Preprocessed" appended.

Example configuration

Below is a sample configuration that starts a custom PowerShell script located on the "PypeServer Documents" share. For this example, the preprocessor is powershell.exe, and the parameters contain the remainder of the command line. The parameters line contains the mandatory {filename} keyword that will be replaced with the actual file name when starting the preprocessor command line.

The name of the preprocessor is "General Construction"

Preprocessor file path

powershell.exe

Note that this is a not the full path to where powershell.exe is installed. In this case the operating system will search for the file in the paths stored in the paths system environment variable.

Parameters

-ExecutionPolicy UnRestricted -File \\PypeServer\PypeServer Documents\PreProcessors\General\PCFReplacePreProcessor.ps1 "\\PypeServer\pypeServer Documents\PreProcessors\General\PCFLookup.csv" {filename}

Parameters breakdown

-ExecutionPolicy Unrestricted

• Sets the PowerShell execution policy for this script runtime

-File \\PypeServer\PypeServer Documents\PreProcessors\General\PCFReplacePreProcessor.ps1

• Full network url file path to the PowerShell script to be run

\\PypeServer\pypeServer Documents\PreProcessors\General\PCFLookup.csv

• Full network url file path to a custom lookup file being used by the PowerShell script

{filename}

• Placeholder keyword that will be replaced by the actual file name during preprocessing