Transforming the Data
One of the most important work associated with the ETL is to calculate absolute profit (Rewards) and loss (Penalties) for the validators. Given the epoch-wise balance for the validators, at the end of every epoch (6.4 Minutes) we calculate the delta between balances of epoch En and En-1 which represents the consensus reward/penalties. One thing to note here is that the balances being considered are BLS Key’s balance and not the validator owner’s balance. These balances reflect the inflation rewards only as the Execution layer rewards are reflected by the balance of the Suggested Fee Recipient set by the Consensus layer node of the validator (Coinbase of the Execution Layer Node).
The difference of balance between two consecutive epochs (En and En-1) can be thus used as the representation of earnings/losses of a validator for epoch En. If the difference between the balances is positive, it's assumed that the validator earned some form of a reward which can be any of the Attestation Rewards, Block Proposal Rewards, etc. These rewards are thus added to the long list of rewards earned by the validator since its activation epoch. In case the difference of balance comes out to be negative, it means that the validator had some penalties because of either missing an Attestation or a Block Proposal. These penalties are added to the list of all the penalties earned by the validator.
Thus at the end of every epoch, we update the rewards and penalties earned by a validator which allows us to track the validator's performance in real-time. One thing to note is that even though rewards and penalties are associated with Slots, the reason we update the rewards every epoch is that the Consensus layer changes state at the end of every epoch only. With Epoch wise performance of the validators, we can model any kind of metric we want.