100 Records Loop Output

I have 100 records in input file if I run graph first time first 20 records will go to output, in next run 21 to 40 records will go to output, like this in 5th run 81 to 100 records will to output. Please let me know how can I achieve this?
Thanks in advance

Showing Answers 1 - 11 of 11 Answers


  • May 6th, 2016

Use partition by round robin component and connect 5 output flows. You can also use next_in_sequence function in Reformat to number the rows and send them to the desired output accordingly.

  Was this answer useful?  Yes


  • May 12th, 2016

Use a lookup file having only one field count and its initial value would be 0.
In filter by expression use next_in_sequence>lookup("lookup_name") and next_in_sequence<=lookup("lookup_name")+20. After every run take count of processed records through rollup and override the lookup file.
So in 2nd run you lookup file value of count would be 20 and in FBE limit would be 21 to 20+20=40.
Or in place of lookup u can use one file and read it value through graph param and use it with next_in_sequence().
There can better approach.

  Was this answer useful?  Yes


  • Jul 27th, 2016

Use Filter By Expression and write next_in
_sequence() < = 20. Then first 20 records are goes to output port. This has to be set as phase 0. From FBE deselect, connect to Reformat and write transform for all fields and connect another output file. Input file name and this output file should be same. This deselect port shold be in phase 1. Let me know any correction plz

  Was this answer useful?  Yes


  • Aug 27th, 2016

next_in_sequence() does not remember the count after first run. There has to be a way where graph can increase the count after every run in a variable and later use that variable to fetch a range of values

  Was this answer useful?  Yes


  • Sep 1st, 2016

Multiple call to next_in_sequence() in same component will not function properly.
i.e: the first function in filter by experssion will evaluate to 1 and second function will evaluate to 2.

  Was this answer useful?  Yes


  • Sep 2nd, 2016

Use a graph level parameters for get the total count of records from output file using shell script interpretation. Then use that parameter in Filter By Expression with next_in_sequence().
FBE condition next_in_sequence()>${PARAM1} and next_in_sequence()<=${PARAM1}

E.g. for first run total record will be zero.

  Was this answer useful?  Yes


  • Jan 22nd, 2017

First make a lookup file with a single column named as count and set it to 0
Read this file in the graph level parameter and name the parameter as count
Now use the reformat component and in the select parameter write the following expression :
Next_in_sequence>=count and next_in_sequence <=count+20
Now suppose in the first run the input file had 100 records
in the transform parameter of the reformat function declare a variable as record_count and keep it increasing by 1 each time a record passes through the reformat component and write it to two output files.
One which will contain the original records and one will keep the count.
Suppose for the first time the record count came as 20 and the graph stops in the next run the graph level parameter will be evaluated and the value will become 20
Now our select expression will look like this
next_in_sequence>=count and next_in_sequence <=count+20
count=20and <=40
this will do the trick :) :p

  Was this answer useful?  Yes


  • Mar 10th, 2017

{i/p file ->FBE(next_in_sequence()<21)->O/p file}Phase 0
{Run program(Sed command to delete 20 lines from ip file)} Phase 1.

  Was this answer useful?  Yes

Eswara Kalyan

  • Mar 29th, 2017

Using Rollup
type temporary_type =
decimal("x01") total_runs = NULL("");
decimal("x01") count = NULL("");
temp :: initialize(in) =
temp.total_runs :: 0;
temp.count :: 0;
out :: key_change(in1,in2)=
out :: in2.ball%6 == 1;
out :: rollup(temp, in) =
out.total_runs :: temp.total_runs + in.run;
//out.count :: temp.count + 1;
out :: finalize(temp, in) =
out.over :: next_in_sequence();
out.run :: temp.total_runs;

  Was this answer useful?  Yes


  • May 16th, 2017

we can use the Partition by range component to segregate the record based on record number

  Was this answer useful?  Yes


  • Apr 17th, 2018

Use the Partition by round robin component and set the block size 20

  Was this answer useful?  Yes

Give your answer:

If you think the above answer is not correct, Please select a reason and add your answer below.


Related Answered Questions


Related Open Questions