...
In Pig 0.8.0 and beyond, you need to set "pig.splitCombination" to false for PigStorageWithInputPath work correctly.
Q: How can I calculate a percentage (partial aggregate / total aggregate)?
The challenge here is to get the total aggregate into the same statement as the partial aggregate. The key is to cast the relation for the total aggregate to a scalar:
| Code Block |
|---|
A = LOAD 'sample.txt' AS (x:int, y:int);
-- calculate the denominator
B = foreach (group A all) generate COUNT(A) as total;
-- cacluate the percentage
C = foreach (group A by x) generate group as x, (double)COUNT(A) / (double) B.total as percentage;
|