What’s coming up in Yellowfin 6.3? Upgrading the broadcast throughput!

Recently, I’ve been working on upgrading the broadcast throughput of Yellowfin.

The existing Yellowfin broadcasting code has several inefficiencies that can lengthen the time needed to deliver a batch of emails to recipients. One of the main problems with this existing code, is in relation to its ability to use multiple cores for increased throughput.

Even though multiple separate broadcasting tasks can run simultaneously on separate cores, each individual broadcast job is restricted to a single thread. This means that if one single broadcast job is running on an 8-core device, only one thread will be utilized. This can lengthen the time taken to finish a job, as Yellowfin will generate a report for each recipient individually (which is in fact another inefficiency of the broadcasting process).

The solution to this problem? Enter new broadcasting code for upcoming release of Yellowfin 6.3 – the next version of Yellowfin’s Business Intelligence software.

A queue of runnable tasks is now created for the each recipient of a report. The queue is then processed, with several tasks processed simultaneously. The threaded nature of this code allows for processing to be spread out across more silicon.

Yellowfin 6.3 broadcast throughput update: Nice Threads….

The actual delegation of code to multiple cores is done by the JVM – and different JVMs may handle this differently. It determines the nature of threaded code, and the code’s dependencies, to determine if it can be split across multiple cores. Keeping this in mind, it is best to write code that has as few shared dependencies as possible. This usually means creating separate Java objects for each thread. This also helps to avoid objects being altered by threads simultaneously, which could cause erroneous results.

Now that that’s done, my next task is to determine how to alleviate the need for running the same report multiple times for each recipient. We currently do this because each user could have individual source filters applied, or require the report in a language other than the default.

Wish me luck…