Scenario:
I am currently having 7 reports that are using one Flat File (.csv) for several charts. Due to new requirements I have to use 4 Flat Files (.csv) in these reports. All 4 Flat Files have the same data column types.

Sounds easy but each chart in all reports needs the data of all 4 flat files.

I know that there are Joint Data Sets but in my opinion it seems kinda awkward to solve this problem with these Joint Data Sets for 4 Flat Files, because the solution would most likely look like this:

Data Set A using Flat File a
Data Set B using Flat File b
Data Set C using Flat File c
Data Set D using Flat File d
Joint Data Set 1. from Data Set A + B
Joint Data Set 2. from Data Set C + D
Joint Data Set from Joint Data Set 1. + 2.

Beside its unhandy solution, my major problem is that I need to do this for 7 reports in total which seems to be a huge time investment.

I am looking for a more efficient solution.

Solution 1: Combining csv files with a scripted data set.

open method:

importPackage(Packages.java.io);
directory = new File(params["fileDirectory"]);  
files = directory.listFiles();
csvfiles = [];
for( i=0; i<files.length; i++ ){
	if( files[i].toString().endsWith(".csv") &amp;&amp; files[i].toString().contains("mydata") ){
		csvfiles.push(files[i].getAbsolutePath());
	}

}  
idx=0;
mx = csvfiles.length;

importPackage(Packages.java.net);
importPackage(Packages.java.io);

inf = new FileInputStream(csvfiles[0]);
inp = new BufferedReader(new InputStreamReader(inf))
inputLine = inp.readLine(); //throw away header
inputLine = inp.readLine(); //throw away header

fetch method:

importPackage(Packages.java.lang);

function openNewFile(){
	inf = new FileInputStream(csvfiles[idx]);
	inp = new BufferedReader(new InputStreamReader(inf))
	inputLine = inp.readLine(); //throw away header
	inputLine = inp.readLine(); //throw away header
	inputLine = inp.readLine(); //get first file
	//if empty csv is encountered go to next file
	if( inputLine == null ){
		inp.close();
		idx++
		openNewFile();
	}
}

var inputLine;
if ((inputLine = inp.readLine()) == null){
	idx++;
	if( idx >= mx ){
		inp.close();
		return false;
	}else{
		openNewFile();
	}
}
var myrowarray =inputLine.split("\\,");
row["col1"] = myrowarray[0];
row["col2"] = myrowarray[1];
row["col3"] = myrowarray[2];

return true;

Solution 2: Combining csv files with esProc SPL script.

The report can be designed in the same way as you would if you were retrieving the data from a database. For detail SPL integration with BIRT, see How to Call an SPL Script in BIRT.

For many difficult calculations of Text Computing, you can refer to Structured Text Computing.

If you have any questions or comments, please leave them below.