Receiving Multiple Files From Execute Command

[DtM]

Hi guys,

I was wondering if anyone can give me any insight into working with the Execute Command configurator?

I've written a Python script which is receiving a CSV list of files, logging into a website and looking for the files in the list.

I then want it to output all of the files back into the flow so that they can be processed.

Running the script outside of Switch works perfectly well using hand typed arguments, but when I start using Switches %1 & %2 arguments it doesn't quite work.

Seeing as I could have multiple returned files, I assumed I would need to use the "Folder at Path" output method in the configurator.

The problem is however, upon interrogation of the values that Switch is passing my script, it would appear that the output path being supplied is still pointing to a single file (in this case the CSV I am supplying) rather a folder.

Does anyone have any experience with this who may be able to shed some light on what I'm doing wrong?

[gabrielp]

Hey man,
I'm not familiar with "Folder at Path" but I can tell you some of the ways I would consider doing it. Within the script, you have to use a sendTo function to route the job out of the script. You want to send the files out as Jobs. Ideally, you want to use the same job, so it keeps the job ticket (private data values, datasets, etc...) for further routing. But, you could also use the Switch class to make new jobs and send them out.

So, if your script knows the location of each file, you can do a for loop and do sendToSingle('the/path/to/the/file') to send each file out of the single outbound connector. You could also use that function to send the entire containing folder as 1 job, but if you wanted to process the files, you'd have to ungroup them later on. To send out the original job file, just do sendToSingle(job.getPath()).

You may want to look at the traffic light sendTo functions like sendToData and sendToLog which will allow you to send different types of files to different places, and supports nice handling of errors or warnings.

[DtM]

Hey man,
I'm not familiar with "Folder at Path" but I can tell you some of the ways I would consider doing it. Within the script, you have to use a sendTo function to route the job out of the script. You want to send the files out as Jobs. Ideally, you want to use the same job, so it keeps the job ticket (private data values, datasets, etc...) for further routing. But, you could also use the Switch class to make new jobs and send them out.

So, if your script knows the location of each file, you can do a for loop and do sendToSingle('the/path/to/the/file') to send each file out of the single outbound connector. You could also use that function to send the entire containing folder as 1 job, but if you wanted to process the files, you'd have to ungroup them later on. To send out the original job file, just do sendToSingle(job.getPath()).

You may want to look at the traffic light sendTo functions like sendToData and sendToLog which will allow you to send different types of files to different places, and supports nice handling of errors or warnings.

Sorry Dominick, I'm a little confused. At the moment I'm just trying to call a straight up Python script that outputs files. As far as the script is concerned it doesn't know anything about Switch, I was just counting on using Switches %1 and %2 arguments in the Execute Command configurator to feed the files where they need to go.

Are you suggesting I write some sort of wrapper in an actual SwitchScript to oversee the whole operation?

[gabrielp]

Oh yeah, I misunderstood your question. My comments were assuming you were writing a script to process.execute and put new files in your flow.

I really know nothing about the execute command configurator :/

[DtM]

Oh yeah, I misunderstood your question. My comments were assuming you were writing a script to process.execute and put new files in your flow.

I really know nothing about the execute command configurator :/

Me neither apparently! Normally I would have done what you suggested and encapsulated the python in a SwitchScript, but was hoping it was simpler to do it this way. It's looking like that's what I'll need to do anyway...

[jan_suhr]

I have run python-scripts in Execute Command with the arguments set up as

Code: Select all

pythonscript.py "$1" "$2"

and out put to File at path

If your Pythonscript pick up one file at the time this would work.

Execute command can be tricky depending on with kind of script you are using and how the input and output is defined by the script. It is also important to have the Switch path variables in quotes. But sometimes that will not work...


Good Luck

Jan