Page 1 of 1

Preserving metadata through Assemble Job element

Posted: Wed Nov 12, 2014 6:32 pm
by jugganaut
Here's the scenario:



Our order system outputs a single XML file per line item of an order. Some orders may have 50 line items, some may have 1. Besides picking up the metadata from these XML files, I also need to perform a count on all line items in an order - I use this number later on to validate downloaded files.



To do the count on the line items (XML Files), I'm assembling the job, after the XML pickup, based on the order number which is derived from a string inside the XML file itself. After assembly, I pass this on to a script that gets a job count. Done. Works fine - at least the count part.



The issue is that after the assemble job element, even with the "Strip Unique name" off, I'm losing the unique metadata - the Assemble Job element is giving the XML files new job IDs. As a quick fix, I could do the XML pickup after the count and disassembly, but I'd like to know if there's a way to just preserve this data.

Preserving metadata through Assemble Job element

Posted: Wed Nov 12, 2014 7:22 pm
by gabrielp
If you leave the unique name, it should have the original metadata if you ungroup them later. Is that not what you're looking for?

Preserving metadata through Assemble Job element

Posted: Wed Nov 12, 2014 10:09 pm
by jugganaut
I'm not really sure what happened, but it appeared that the Assemble Job element was clipping the first underscore from the job ID. I built a test flow just to try out ungroup verses dismantler and it worked fine with both. Made sure all settings were the same on the main flow, but it was still clipping off that first underscore. So I deleted that Assemble element and copied in the one from the test flow and now it's working correctly.



Does switch ever delete the first underscore of the job ID for any reason?

Preserving metadata through Assemble Job element

Posted: Wed Nov 12, 2014 11:02 pm
by jugganaut
Nevermind - it's not working. Thoroughly confused now.

Preserving metadata through Assemble Job element

Posted: Thu Nov 13, 2014 3:01 pm
by freddyp
It is not clear to me what you are doing exactly. List the elements by name in the order that you use them and mention any relevant properties, in this style:



1: Input folder. 2: "XML pickup" with "Pickup mode - Metadata is asset". 3: "Assemble job" with "Scheme - Custom" - "Number of jobs - sth out of the metadata" - "Merge metadata - No" etc.



And how do you get the PDF's: "Inject job" or a download using a script? And when?



What is the final output supposed to be? A single PDF?



Freddy

Preserving metadata through Assemble Job element

Posted: Thu Nov 13, 2014 3:58 pm
by jugganaut
Here's what I've got:



My input files are, as I've mentioned, XML files that represent line items in an order.



1. XML files are input into the flow

2. XML Pickup as asset

2.5 Route non-print items out of flow. Using XML data to determine this.

*** we're assembling job to get a file count we'll use later on to verify the correct number of downloaded files ***

3. Assemble Job element

Custom scheme:

Job identifier: job number picked up from XML

Number of files: 500 (could be 1, could 500)

After 1 minute

Merge Metadata: no

Job folder name: same as job identifier

Strip unique name: no

Duplicates: overwrite

4. Rename element

remember original name: yes

private data key: original

files inside job folder: leave alone

5. Script element to retrieve file count - puts into private data

6. Ungroup Job (have also tried dismantle)

7. Goes through another script to retrieve some data from metadata and put into private data

8. routes files that have no download link to an email element

9. Goes through another script to parse a download link from metadata

10. Downloads file through cURL command and renames file from private data elements retrieved earlier:

Concurrent execution mode

Result in folder

Copy input job: no

11. Dismantle element to strip enclosing folder form cURL command

12. Assemble element where we use the count retrieved earlier to verify downloaded files

job identifier: private data key: original

number of files: script expression

after 15 minutes

job folder name: private data key: original

strip unique name: yes





The end result needed is a job folder named to the job number, and inside it will contain the downloaded files. The issue is that the unique metadata from the XML files gets overwritten when the XML files get assembled into the job folder in the first Assemble Job element.



Basically the problem is that you can't have jobs within a job, or at least not the way I'm doing it.



So if there's no way to preserve metadata in "subjobs", then I will just do the XML pickup after the first assemble job element and for the routing out of non-print items in step 2.5 I'll use regex or something...

Preserving metadata through Assemble Job element

Posted: Thu Nov 13, 2014 4:13 pm
by jugganaut
What's happening now is that when the XML files get to the cURL command, it's download the same image for all XML files because the metadata has been overwritten...

Preserving metadata through Assemble Job element

Posted: Thu Nov 13, 2014 5:38 pm
by gabrielp
So frustrating. Just typed out a whole response and Crossroads-world "half" logged me out and it didn't let me post it or copy my text and then redirected me to the homepage. Then I tried posting again and got the redirect loop issue. These forums are so buggy, it's honestly problematic. Doesn't Enfocus make software? Fix the forums!



I experienced the same thing in my test. Here is an example flow with a solution to fix it: https://github.com/dominickp/Switch-Ass ... nd-Packing



Put a file in each source folder. The flow on top will route both jobs into the folder at the end where only one should go. This is because assemble seems to only honor the value of the first job it sees in the assemble process.



On the bottom if you do that, you'll see they are routed properly because I've introduced a pack and unpack before and after the assemble/ungroup which preserves the individual metadata from whatever the assemble is doing to them.

Preserving metadata through Assemble Job element

Posted: Thu Nov 13, 2014 7:02 pm
by jugganaut
I've begun to make a habit of copying my responses before I post - I've lost a handful of thoughtful responses and understand the pain! And why not use something like phpbb?



I've got to work through some paper pushing this morning, but I look forward to testing out your solution. Thanks!

Preserving metadata through Assemble Job element

Posted: Fri Nov 14, 2014 1:28 pm
by freddyp
After the "Ungroup job" you have the individual XML files again, but the metadata is gone as a result of the "Assemble job" that came before. Insert another "XML pickup" with "Metadata is asset". I think this will allow you to get to the desired result.



Another remark: you do not need a script to count the number of files in a job. [Job.FileCount] will give you that too.



Freddy