SAP XI FREQUENTLY ASKED QUESTIONS

Symptom

You are experiencing problems with the SAP XI 3.0 / PI 7.0 File Adapter and/or need further detailed information about its operation and configuration.
Other terms

Exchange Infrastructure, XI30, XI70, Adapter Framework, Frequently Asked Questions, HOWTO
Reason and Prerequisites

This note serves as FAQ document for the SAP XI 3.0 / PI 7.0 File Adapter and contains answers to the most commonly asked questions about the adapter's operation and configuration. It will be updated with new information from time to time as the need arises.
Solution
Table of Contents

1. Information to Include in OSS Message

2. Read-Only File Processing

3. File Locking

4. FTP Sender File Processing in Cluster Environment

5. File Receiver: Overwrite Protection

6. Acknowledgements

7. File Construction Mode "Add Time Stamp" and "Add Counter" in Cluster Environment

8. FTP Adapter Timeout Configuration

9. Module Development: Accessing the Source File Name from a Module

10. .fieldContentFormatting=nothing and SXMB_MONI

11. File Sender: Processing Multiple Source Directories

12. File Sender: Quality of Service EOIO (Exactly Once In Order)

13. File Receiver: FTP Problems for Large Files with Microsoft IIS

14. Memory Requirements

15. Operating System Command

16. File Encoding

17. Empty File Encoding Parameter

18. File Sender: Scheduling

19. Secure FTP (FTPS)

20. File Sender: Special Characters in File Names

21. File Sender: Performance Issues After Configuring A Large Amount Of Channels

22. File Sender: File Modification Check For Transport Protocol "FTP"

23. FTP: Active Data Connection Support

24. FTPS: Server certificate rejected by ChainVerifier

25. FTP Proxy Support

26. FTP Connection Problems

27. FTP Directory Listing Semantics

28. sftp Support

29. File Content Conversion fieldFixedType Parameter Not Working

30. Missing Features After Upgrade

31. Empty File Processing

32. File Construction Mode "Add Counter": Counter Reset

33. FTPS: "AUTH TLS" Command Unrecognized

34. FTP: Directory Specification / CWD Command

35. File Sender Error Handling

36. FTP: Adapter Polls More Frequently Than Configured
Questions and Answers

1. Information to Include in OSS Message

* Q: I need to open an OSS message regarding the File Adapter with SAP. Which information should I put into the message?

* A: Refer to note 854536.

2. Read-Only File Processing

* Q: I have configured a File Adapter sender channel to process a source file to which it only has read-only access (e.g., because it has been written by a different OS user). The file is not processed, but the channel is nevertheless listed as working in the Adapter Monitoring. Have I encountered a bug?

* A: This behavior is by design: The File Adapter sender channel does not process read-only files by default. As this is intended, you will never see an error message indicating this condition in the Adapter Monitoring or log. To enable processing of read-only files, set the "Files Locked (Read-Only)" flag for the respective sender channel.

3. File Locking

* Q: Does the File Sender Adapter honor if another process has locked a file for exclusive use?

* A: Unfortunately, the J2EE 1.3 technology the File Adapter is built upon does not support file locking. This limitation affects the File Adapter's operation. Depending on whether the JRE implementation for the operating system under which the adapter runs uses mandatory file locking or advisory file locking, opening a file that is currently being written to by another process will fail or not.

If opening the file fails, no problem exists and the adapter will try to open the file each poll interval until it succeeds.

However, if opening the file is not prevented by the operating system, the adapter starts to process the file although it is still being modified. Since XI 3.0 SP11 / PI 7.0 there is a parameter named "Msecs to Wait Before Modification Check" in the advanced settings of the File Sender channel configuration to work around this issue. This setting causes the File Adapter to wait a certain time after reading, but before sending a file to the Adapter Engine. If the file has been modified (which is basically determined by comparing the size of the read data with the current file size of the input file) after the configured interval has elapsed, the adapter aborts the processing of the file and tries to process the file again after the retry interval has elapsed.

4. FTP Sender File Processing in Cluster Environment

* Q: When running the File Adapter in an environment with more than a single J2EE server instance, the same file is fetched by multiple cluster nodes at once. How do I solve this problem?

* A: Initially, make sure that you do NOT use the processing mode "Test" for your File Sender channel. This mode is intended to be used only for testing purposes and is not guaranteed to work in a cluster environment.

Additionally, activate the "Advanced Mode" for the respective sender channel or upgrade to XI 3.0 SP11 Patch Level 2 or XI 3.0 SP12. PI 7.0 is not affected. No further configuration changes are necessary.

5. File Receiver: Overwrite Protection

* Q: When running a File Adapter Receiver <= XI 3.0 SP9 with a File Construction Mode "Create", target files are overwritten although the "Overwrite Existing Files" setting is not active in the Receiver channel. How do I solve this problem?

* A: This problem has been corrected with XI 3.0 SP10. PI 7.0 is not affected.

6. Acknowledgements

* Q: Does the File Adapter support acknowledgements?

* A: You need to distinguish system acknowledgements (indicating that a message has been received by the target system) and application acknowledgements (indicating that the message has been successfully processed by the application on the receiver side).

The receiver of an XI message will only send an acknowledgement back to the sender if the sender has requested one. However, the File Adapter has no functionality that relies on the receipt of an acknowledgement, so it never requests one.

On the other hand, if a File Adapter Receiver receives a request to send an acknowledgement, it will do so for a system acknowledgement request. Application acknowledgements are not supported at all as the File Receiver has no way to determine if the written file has been correctly processed by the back-end application, which is what a positive application acknowledgement would imply.

7. File Construction Mode "Add Time Stamp" and "Add Counter" in Cluster Environment

* Q: We have set up a File Adapter Receiver on an XI installation with multiple server nodes. The receiver channel is configured to use the File Construction Mode "Add Time Stamp" or "Add Counter". We have noticed, that sporadically two messages are written to the same file name. How can we work around this issue?

* A: If you are using the transport protocol "File System (NFS)", apply note 869234. For the transport protocol "File Transfer Protocol (FTP)" the time stamp and counter is not guaranteed to be unique across cluster nodes, so two messages delivered to the same receiver channel at the same time on different cluster nodes might cause the target to be overwritten by the second message. If you need to create truly unique files by the receiver, use the File Construction Mode "Add Message ID".

8. FTP Adapter Timeout Configuration

* Q: We are using the FTP Adapter to send or receive messages from/to an FTP server, which is sporadically experiencing connectivity problems. This causes the FTP Adapter to hang. Is there any way to configure a timeout?

* A: Refer to note 849089.

9. Module Development: Accessing the Source File Name from a Module

* Q: When processing a message originating from a File Adapter Sender channel, how do I access the original name of the file read by the File Sender channel from within a module?

* A: See note 819761 for a HOWTO.

10. .fieldContentFormatting=nothing and SXMB_MONI

* Q: We have configured a File Sender Adapter with File Content Conversion not to remove the whitespace around fixed-width fields from the input file by setting the parameter .fieldContentFormatting to "nothing". Nevertheless, the whitespace seems to be trimmed as we can see in SXMB_MONI. Have we encountered a bug?

* A: SXI_MONITOR displays an interpreted version of the XML payload, which unfortunately hides the whitespace. Nevertheless, this whitespace is still present in the generated XML file as you will see if you send the message to a File Adapter Receiver that writes the payload to a file.

11. File Sender: Processing Multiple Source Directories

* Q: Can the File Sender Adapter be configured to poll multiple source directories from a single channel?

* A: This functionality is available starting with XI 3.0 Support Package 14 as well as PI 7.0.

12. File Sender: Quality of Service EOIO (Exactly Once In Order)

* Q: How does a File Adapter sender channel handle the quality of service EOIO? Are files that match the File Name Scheme in the sender's channel configuration processed in arbitrary order?

* A: The File Adapter allows you to configure the Processing Sequence of files for the Transport Protocol "File System (NFS)", which also determines the order for EOIO processing. Files can either be processed in ascending alphabetical order (Processing Sequence "By Name") or by their last modification time stamp (Processing Sequence "By Date"), where the oldest file is processed first. For the Transport Protocol "File Transfer Protocol (FTP)" files are always processed in ascending alphabetical order.

13. File Receiver: FTP Problems for Large Files with Microsoft IIS

* Q: I have configured a File Adapter receiver channel to upload to a Microsoft Internet Information Server FTP server using the "Put File" mode "Use Temporary File".
When sending a large file to the receiver, I get the following error: "com.sap.aii.adapter.file.ftpFTPEx: : The process cannot access the file because it is being used by another process."
How do I solve this problem?

* A: The cause of this problem is a known bug in the Microsoft IIS FTP server implementation you are using. For details, refer to:

o IIS 4.0: http://support.microsoft.com/default.aspx?scid=kb;en-us;238644
There is no fix available from Microsoft. As a workaround, set the configuration parameter "Put File" to "Directly".

o IIS 5.0: The problem is currently not documented by Microsoft. As a workaround, set the configuration parameter "Put File" to "Directly".

o IIS 6.0: http://support.microsoft.com/default.aspx?scid=kb;en-us;828086
Apply the hotfix provided by Microsoft.

14. Memory Requirements

* Q: Which memory requirements does the File Adapter have? Is there a restriction on the maximum file size it can process?

* A: The maximum file size that can be processed by the File Adapter depends on a number of factors:

o The most important one is the size of the Java heap, which is shared among all messages processed at a certain point in time. In order to be able to process larger messages without an out of memory error (OOM), it is recommended to increase the size of the available Java heap and/or to reduce the concurrency in the system so that fewer messages are processed in parallel.

o Another factor negatively influencing the maximum message size in releases up to and including XI 3.0 SP 13 is an enabled charcter set (encoding) conversion if the message type is set to "Text".

o Using the transport protocol "File Transfer Protocol (FTP)" also uses more memory for processing than the transport protocol "File System (NFS)" (up to and including XI 3.0 SP 13).

o If the Message Protocol "File Content Conversion" is used in a File Sender channel, consider that not only the size of the input file affects the File Adapter's memory usage, but even more the size of the XML resulting from the conversion, which is usually a few factors larger than the original plain text file.

To reduce the memory consumption in this scenario, consider configuring the setting "Maximum Recordsets per Message" for the sender channel. This will cause the input file to be split into multiple smaller mesages.

15. Operating System Command

* Q: I am having difficulties getting an external operating system command to work. What can I do to diagnose the problem?

* A: Refer to note 841704.

16. File Encoding

* Q: How do I correctly configure the File Encoding used by the File Adapter?

* A: Generally, the message payload of an XI message is treated as UTF-8 when it needs to be converted from or into a different encoding. So, what you specify in the File Sender channel is the source encoding for an encoding conversion to UTF-8. On the other hand, in the Receiver channel, you specify the target encoding for a conversion from UTF-8. If you configure a channel's File Type setting as "Binary", no conversion will be applied.

Depending on your scenario, only some encoding settings actually make sense and lead to the expected results:

o XML Files
An XML file's encoding is set in the XML header, which is later interpreted when parsing the XML, e.g. in the mapping, so there is no necessity to perform an encoding conversion in the File Adapter. As a rule of thumb, always configure the File Type parameter of a sender or receiver channel as "Binary" when reading or writing XML data.
Important: Even if you configure a File Encoding in the File Adapter channel, the File Adapter will not re-write the XML header to reflect the changed encoding, so you will probably see an XML parsing error later during the processing of the message if you specify an encoding.

o Flat Files with File Content Conversion
For a File Sender channel, configure the encoding of the source file. The file will be interpreted according to the configured encoding and converted to XML with an UTF-8 encoding.
For a File Receiver channel, configure the encoding to match the encoding you would like to be written to the target flat file.

o Flat Files without File Content Conversion
Whether to configure an encoding in this case depends on if you want to pass through the file "as is", e.g. within a File Sender to File Receiver scenario, or if you want to convert the file's encoding on its way through the Integration Server. For "as is" processing, configure both the sender and the receiver using the File Type setting "Binary".
To apply an encoding conversion, configure the respective source and target encoding in both the sender and receiver channel.
Important: Configuring an encoding in the receiver channel will only lead to the expected results if the payload sent to the receiver channel is in UTF-8 format (e.g., by having specified an encoding conversion in the Sender channel).

17. Empty File Encoding Parameter

* Q: Which encoding is used if I configure the File Type as "Text", but leave the File Encoding parameter empty?

* A: In this case, the system encoding will be used, e.g. "Cp1252" on most western Windows installations.

18. File Sender: Scheduling

* Q: How do I configure a File Adapter sender channel to poll a directory at a specific time of day?

* A: This is currently not possible. However, we are evaluating whether we can provide a channel scheduling functionality in a future release.

19. Secure FTP (FTPS)

* Q: I would like to use Secure FTP (FTPS) with the File Adapter. Does my FTP server support this out of the box? Which configuration changes do I need to apply to my FTP server?

* A: Not every FTP server supports FTPS and many that do require a configuration change to activate the FTPS protocol extension. Contact your FTP server software vendor for detailed information.

20. File Sender: Special Characters in File Names

* Q: I am trying to get the File Adapter to poll a file, which contains special characters (e.g., accented characters or umlauts) in its file name. However, irrespectively of the wildcard mask I specify in the File Adapter sender channel configuration, the file does not get picked up. Which configuration setting do I need to change to get my scenario working?

* A: Under certain operating system platforms, such as Solaris, the APIs used by the Java Runtime (JRE) are not Unicode-aware. Consequently, the JRE needs to be configured to correctly interpret the character set it receives from the operating system.
This is configured through the "file.encoding" system property as well as the "LANG" environment variable.
Make sure you set "file.encoding" to a character set (such as ISO-8859-1) that supports the special characters you would like to process. This system property can be configured by appending "-Dfile.encoding=" to the Java VM parameters section of the SAP J2EE Config Tool.
Additionally, you need to set the "LANG" environment variable to a locale that supports more than 7 bits, such as "de.ISO8859-1". The encoding you specify in the LANG environment variable needs to match the encoding set via "file.encoding".
You can persistently configure the environment variable by setting it in the profile $HOME/.sapenv_$HOSTNAME.csh of the adm user: setenv LANG de.ISO8859-1

21. File Sender: Performance Issues After Configuring A Large Amount Of Channels

* Q: After configuring a large amount of File Adapter sender channels, the J2EE Engine becomes very slow and some services start to block. How can I solve this issue?

* A: Up to and including XI 3.0 SP13 each File Adapter sender channel permanently consumes a J2EE application thread. To solve this issue, increase the number of configured J2EE application threads using the SAP J2EE Engine Config Tool ("cluster-data" -> "Global server configuration" -> "managers" -> "ApplicationThreadManager" -> "MaxThreadCount").
Starting with XI 3.0 SP14 / PI 7.0 application threads are allocated on demand by the File Adapter and returned to the thread pool after it has finished the polling sequence, so thread shortage situations will typically occur much more rarely than with earlier SPs.

22. File Sender: File Modification Check For Transport Protocol "FTP"

* Q: Is the file modification check (advanced mode option "Msecs to Wait Before Modification Check") in a File Adapter sender channel supported for the transport protocol "File Transfer Protocol (FTP)"?

* A: No, this option is not supported for the transport protocol "File Transfer Protocol (FTP)", but only for the transport protocol "File System (NFS)".

23. FTP: Active Data Connection Support

* Q: Does the File Adapter support active data connections when connecting to an FTP server?

* A: Up to and including XI 3.0 SP14, the File Adapter exclusively uses passive data connections. Starting with XI 3.0 SP15 / PI 7.0, active connections are available.

24. FTPS: Server certificate rejected by ChainVerifier

* Q: After configuring a File Adapter FTPS communication channel, I see the following error message in the Adapter Monitor: iaik.security.ssl.SSLException: Server certificate rejected by ChainVerifier. What do I need to configure to get my scenario working?

* A: The File Adapter checks whether a trusted certificate chain exists for the X.509 certificate of the FTP server it is connecting to. A certificate chain is considered trusted if the chain is valid and at least one of its certificates is directly trusted. If the certificate chain is untrusted, the above exception is thrown. To configure a single certificate or a certification authority's CA certificate as directly trusted, add the respective X.509 certificate to the "Trusted CAs" view of the "Key Storage" service in the SAP J2EE Visual Administrator. In most cases, this will be the CA certificate that was used to sign the server certificate.

25. FTP Proxy Support

* Q: Does the File Adapter provide a means to configure an FTP proxy server used to establish any FTP connections?

* A: No, this is not supported.

26. FTP Connection Problems

* Q: I have configured a File Adapter channel to connect to an FTP server, but receive one of the following error messages in the adapter monitor:

o Error connecting to ftp server '': SocketException: Connection reset

o Error connecting to ftp server '': SocketException: Connection refused

o Error connecting to ftp server '': ConnectException: Connection timed out

* A: This problem is either caused by incorrect firewall / packet filter settings or an incorrect configuration of the FTP server. Also make sure that you have correctly specified the host name / IP address and port of the FTP server.
Changing the connection type from 'active' to 'passive' (or vice versa) might additionally help to work around the incorrect firewall configuration.

27. FTP Directory Listing Semantics

* Q: Which semantics does the File Sender Adapter apply to determine which files to retrieve from an FTP server?

* A: Up to and including XI 3.0 SP13 the File Adapter will issue an NLST command (see RFC 959) with the configured source file name scheme as first parameter to the FTP server. It will try to retrieve any of the files returned by the FTP server in response to this command.
Starting with XI 3.0 SP14 / PI 7.0, the File Adapter will issue a LIST command with the configured source file name scheme as first parameter to the FTP server. If it is able to parse the returned result (which is the case for UNIX- and Windows-style directory listings), it will retrieve any of the returned files. If it cannot parse the result, it will fall back to the pre-SP14 implementation and use the NLST command as described above.

28. sftp Support

* Q: Does the File Adapter support the sftp (File Transfer over SSH) protocol?

* A: No, this protocol is not supported and there are currently no plans to implements sftp support in the File Adapter.

29. File Content Conversion fieldFixedType Parameter Not Working

* Q: After configuring a File Adapter channel with File Content Conversion and the fieldFixedType=byte parameter set, I noticed that this setting does not seem to have any effect.

* A: The parameter's name should correctly read "fieldFixedLengthType". The documentation will be updated accordingly with XI 3.0 SP17 / PI 7.0.

30. Missing Features After Upgrade

* Q: After applying a support package, I am unable to find a new feature advertised in the release notes in the channel configuration.

* A: You most likely did not import the updated SAP_BASIS content for the support package.

31. Empty File Processing

* Q: How does the File Adapter handle empty files and messages with a zero-sized payload?

* A: Empty files in the sender adapter will not trigger the creation of a message. If an empty message is sent to a receiver channel, no output file will be created.

32. File Construction Mode "Add Counter": Counter Reset

* Q: After configuring a File Adapter receiver channel with the File Construction Mode set to "Add Counter", I noticed that the counter gets reset each time I restart the J2EE engine.

* A: This behavior is by design. The current counter value is not persisted, but only kept in memory. It is intended to generate distinct file names, but not strictly increasing numbers. After a J2EE restart, the File Adapter will start writing messages beginning with the smallest free counter number (skipping existing larger numbers).

33. FTPS: "AUTH TLS" Command Unrecognized

* Q: After configuring a File Adapter channel with the Transport Protocol "FTP" and connection security set to "FTPS", I receive the following error in the adapter monitoring: Syntax error, command unrecognized: 'AUTH TLS'
What is causing this?

* A: Your FTP server implementation does not implement FTPS as specified in RFC 4217. For details, contact your FTP server vendor.

34. FTP: Directory Specification / CWD Command

* Q: Is there any way to configure a File Adapter channel with the Transport Protocol "FTP" without an explicit source / target directory, so that no CWD command will be issued?

* A: The directory specification is mandatory. According RFC 1123 an FTP server is required to support the CWD command.

35. File Sender Error Handling

* Q: I have configured a File Sender channel with File Content Conversion or a custom-developed module, which may throw an exception. When the File Adapter encounters an invalid file, which triggers a conversion error or an exception in a module, the adapter will enter a retry interval and upon each following processing attempt try to process the faulty file. This basically prevents the File Adapter from picking up files that are located after the fauilty file according to the configured sort order. How can I change this behavior?

* A: This is by design. When the File Adapter encounters an invalid file, manually remove this file from the configured source directory. You can configure an alerting rule to get notified each time this occurs.
We are currently evaluating possible solutions that could be used to skip this manual interaction.

36. FTP: Adapter Polls More Frequently Than Configured

* Q: I have configured a File Adapter Sender channel with the transport protocol "FTP". When no file is present in the configured input directory, the adapter connects to the FTP server at least once a second or even more frequently. What is causing this?

* A: Most likely, this behavior is caused by an implementation error in the FTP server, which you are connecting to:

The FTP protocol specification (RFC 959) requires that when issuing a NLST command to retrieve a file list for a given wildcard pattern a list of valid file names gets returned. Error messages must be transmitted via the control channel. From the specification:

"NAME LIST (NLST)

This command causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. The server will return a stream of names of files and no other information. The data will be transferred in ASCII or EBCDIC type over the data connection as valid pathname strings separated by or . (Again the user must ensure that the TYPE is correct.) This command is intended to return information that can be used by a program to further process the files automatically. For example, in the implementation of a "multiple get" function."

However, your FTP server returns a human-readable error message in the data connection instead of a file list when no files match the wildcard pattern (the channel's file name mask) provided to the NLST command, which is - in compliance with RFC 959 - interpreted as a file name by the File Adapter.

An example FTP session:
[...]
ftp> ls file*.txt
200 PORT command successful.
150 ASCII data connection for /bin/ls
file*.txt: No such file or directory
226 ASCII Transfer complete.
[...]

The line causing the problems reads:
file*.txt: No such file or directory

As this line is interpreted as a file name by the File Adapter (as per RFC 959), the File Adapter will try to retrieve this file, which will - of course - fail. However, as the FTP server has returned at least one (although invalid) file name, the File Adapter will immediately poll again for new files. This is by design and cannot be configured.

9.Known Problems and Incompatibilities

10.JDBC Adapter: Type 2 JDBC Driver Deployment