Slowing down flows

jugganaut · Post by **jugganaut** » Tue Nov 04, 2014 10:11 pm

I have a flow that takes in XML files and processes them, however I'm dumping in 50-100 files at a time. When this happens, the script I'm using gets choked up and outputs incorrect data - I just need to slow it down. Is there a method to release a file every 10 seconds or something like that?

dkelly · Post by **dkelly** » Tue Nov 04, 2014 10:18 pm

Hold Job element or lower the number of maximum tasks in Preferences

jugganaut · Post by **jugganaut** » Tue Nov 04, 2014 11:32 pm

Got it. I tried using the Switch.Counter to release one item every 10 seconds, but it doesn't work well - it will release 5 or 6 at a time every 10 seconds.

Is there a better way to release 1 item every 10 seconds?

gabrielp · Post by **gabrielp** » Wed Nov 05, 2014 12:01 am

Yeah I don't think hold jobs is smart enough to let them go sequentially.

Perhaps if you had a release condition being a completed job returning and releasing a previous one. But what would release the first job?

This would be easy if you could have a connector determine a location based on the number of jobs in the following folder. Like, send a file from folder a to folder b, if folder_b.count == 0. But I don't think you can do that outside of a script.

What about a simple script that had a setTimeout of a few seconds? I think the way Switch works. That seems like it would work if every file needed to pass through it and Switch launched only one instance of your script per flow -- not sure if that is the case but I know only a single instance of the SmartStream VDP configurator can run at a time, so maybe.

ArielRauch · Post by **ArielRauch** » Wed Nov 05, 2014 7:49 am

I am not in front of switch right no but you can decide whether your script runs serialized or may work parallel for different instances.

Have you tried to play with this?

ArielRauch · Post by **ArielRauch** » Wed Nov 05, 2014 7:51 am

another approach would be to build a kind of semaphore with global data

freddyp · Post by **freddyp** » Wed Nov 05, 2014 9:08 am

In the properties of the script in Switch Scripter you can set whether it executes concurrently or serialized. It seems the choice for you is to set this to "Serialized".

I do not know what the script does, but if it stays within the confines of Switch internal stuff (no Process.execute, no temporary files) I find it odd that the execution of the script should yield different results depending on how many files there are in the input folder. So I do not want to jump to conclusions, but I do not rule out a programming error.

Freddy

gabrielp · Post by **gabrielp** » Wed Nov 05, 2014 3:12 pm

freddyp wrote: In the properties of the script in Switch Scripter you can set whether it executes concurrently or serialized. It seems the choice for you is to set this to "Serialized".
ArielRauch wrote: you can decide whether your script runs serialized or may work parallel for different instances.

Very good to know. Thanks

dkelly · Post by **dkelly** » Thu Nov 06, 2014 10:15 pm

Here's a script that implements simple and reliable "hold job" where you specify how many jobs to release every N minutes.

https://www.dropbox.com/s/zb7i6iy8cn9n2 ... cript?dl=0

Dwight Kelly

Apago, Inc.

For all of your Switch flow and scripting development needs

jugganaut · Post by **jugganaut** » Fri Nov 07, 2014 4:04 pm

dkelly,

Thanks for that script! After some research I realized it was a command line element that was jamming things up because it was set to concurrent - switching it to serialized cured the problem...

But I have another flow where this Hold script will be useful...