Credit goes to Claude 3.5 Sonnet!
Let's consider a GPU using watts and capable of TFLOPS (trillion floating-point operations per second) in FP16.
First, we convert the GPU's performance from TFLOPS to operations per second:
Then, we calculate operations per hour:
We convert the GPU's power consumption to kWh:
Finally, we calculate the GPU's efficiency in ops/kWh:
The brain has an average of trillion synapses. Let's assume each synapse switches times per second and computes a function that corresponds to neural-network weight. The brain consumes approximately watts of power.
We start by calculating the brain's total operations per second:
Then, we calculate operations per hour:
We convert the brain's power consumption to kWh:
Finally, we calculate the brain's efficiency in ops/kWh:
The ratio of brain efficiency to GPU efficiency is:
This means the brain is performing times more operations per kWh than the GPU.