This project has moved and is read-only. For the latest updates, please go here.

Usage scenarios of AfterGet DoNothing

Jun 11, 2015 at 10:14 AM

I'd like to discuss usage scenarios for setting AfterGet action to DoNothing. This is a feature that has not been properly documented AFAIK and is not very useful to me as it is currently implemented.

I have a scenario where some files are being distributed to multiple parties (one of which is my organization) in the same SFTP folder. Consequently we do not have the permission to modify the files at target location. I have set the AfterGet to DoNothing, however the SFTP adapter just retries to get the files at every polling interval. I have put a duplicate detector in the receive pipeline, but it still generates a lot of duplicate warnings.

That's why I propose to modify the SFTP adapter to keep the transferred files in the list of files in process if AfterGet is set to DoNothing. This way the adapter would not re-get the files on each polling interval until the next host instance restart.

Would the proposed change break somebody else's scenarios? Please let me know how you are using this feature.

Best regards, Igor
Jun 29, 2015 at 8:20 AM
I'm not sure I understand your scenario, but as I understand it you cannot neither move a read file, nor delete it?
In that case, someone else has to delete or archive the file eventually, otherwise the folder will have too many files to be practically manageable.
I would suggest that you implement a new AfterGet action, you might call it "IgnoreInSubsequentReads".
That action can add the filename to a global list, perhaps in a database, with filenames to ignore when getting files.

How does that sound?

BR, Peter
Jun 29, 2015 at 12:26 PM

Yes, the SFTP folder only contains the last 5 weekly files. This is beyond my control and is managed by the provider of these files.

We already have an AfterGet action called "DoNothing", which in its current implementation does not work very well, because it continuously fetches the same files. Would it not be best to improve this functionality? I agree with you that it would be ideal to record this information in the database.

Best regards, Igor
Jun 30, 2015 at 1:18 PM
Well, the AfterGet action DoNothing does its job very well, namely, nothing - as compared to Rename or Delete.

I think there can be others interested in what you want. The source code is available for download; just go ahead and implement your new action and if you want, upload your changes as a patch!

BR, Peter
Jun 30, 2015 at 1:29 PM
Another solution springs to mind. You can implement a duplicate check in a receive pipeline with a custom pipeline component that checks the filename against a database lookup. Not the most elegant solution, but workable if you don't want to change the adapter code.