feat: add a stat example to verify impl vs theoretical
This commit is contained in:
@@ -87,4 +87,43 @@ Now it's similar to xxhash where you pass in a void pointer and a length to it.
|
||||
|
||||
## Fixing clangd
|
||||
|
||||
Post the split, I needed to fix clangd as it did not
|
||||
Post the split, I needed to fix clangd as it did not register variables across files as I did not
|
||||
have a valid cmake. So added that and ran `cmake -S . -B build` and all worked fine.
|
||||
|
||||
---
|
||||
|
||||
# The Math
|
||||
|
||||
Better to quickly go over the math as well
|
||||
n - no of keys added ( expected ), m -> filter size in bits, k -> no of hash functions
|
||||
|
||||
Now there are m bits, and consider one insertion, and just one hash function
|
||||
Proba that it's still 0 is (1-1/m) as 1/m is the chance that bit is occupied and hence 1.
|
||||
Now there are n keys and k such hash functions. This takes you to
|
||||
|
||||
(1-1/m) ^ nk
|
||||
|
||||
and then P(some particular bit is set) is 1-(1-1/m)^nk
|
||||
|
||||
For a test k, it's false positive if all of the k bits are set which gives you
|
||||
|
||||
(1 - (1-1/m)^nk ) ^ k
|
||||
|
||||
For large m, (1-1/x)^x ~= 1/e
|
||||
so the inner part becomes 1 - e^(-kn/m)
|
||||
|
||||
This last step is incorrect as the assuming some bitstates to be independent
|
||||
P(bit A is 1 and bit B is 1) = P(bit A is 1) * P(bit B is 1)
|
||||
|
||||
Now this looks correct except that it is not
|
||||
https://www.math.umd.edu/~immortal/CMSC420/notes/bloomfilters.pdf
|
||||
|
||||
But that's besides the point for now.
|
||||
|
||||
## My error probab function?
|
||||
|
||||
Based off of Guava, it uses the actual free bits at runtime to give a "live" estimate
|
||||
bits filled fraction is P(any arbitrary bit is 1)
|
||||
False positive = all of those bits end up being 1 for some hash = P^k
|
||||
|
||||
This has the same independence issue in assumption though.
|
||||
|
||||
Reference in New Issue
Block a user