feat: add bloom_filter lib files using xxhash64 internally
This commit is contained in:
@@ -22,3 +22,69 @@ This double hashed func almost behaves like a random number generator.
|
||||
## std::hash
|
||||
|
||||
you make a hasher object and them immediately call it
|
||||
|
||||
## Designing the api
|
||||
|
||||
Now that I have a basic version ready I need to design some sort
|
||||
of an api going forward - for the header. For now I'm sticking
|
||||
to the traditional impl.
|
||||
|
||||
There was a google's impl
|
||||
https://github.com/google/guava/blob/master/guava/src/com/google/common/hash/BloomFilter.java
|
||||
|
||||
and also another cpp lib that's around 10y old
|
||||
https://www.partow.net/programming/bloomfilter/index.html
|
||||
|
||||
Ok going step by step, first the hash function
|
||||
|
||||
### Hash func
|
||||
|
||||
The guava one uses MurmurHash3 ( first I've heard of it )
|
||||
There are two sorts of hashes - cryptographic and non crypto
|
||||
|
||||
The diff is explained here: https://security.stackexchange.com/questions/11839/what-is-the-difference-between-a-hash-function-and-a-cryptographic-hash-function
|
||||
|
||||
The point being - cryptographic ones ensure security against some
|
||||
adversary and guarantee some stuff like
|
||||
1. preimage resistance - given hash find message
|
||||
2. second preimage resitance, given just x and h(x), find another x'
|
||||
with same hash
|
||||
3. collision resistance - free to choose any pair but need to match
|
||||
hashes
|
||||
|
||||
When you loosen these constraints - faster hashes are possible.
|
||||
|
||||
In any case - I googled and seems like xxhash is the fastest now
|
||||
so I'll use that.
|
||||
|
||||
From there I downloaded the simplest impl
|
||||
https://create.stephan-brumme.com/xxhash/ the v2 version for 64.
|
||||
It's a single header so that should be good enough.
|
||||
|
||||
### Back to actual api?
|
||||
|
||||
I liked Google's api for it - especially the naming and all.
|
||||
I won't b supporting any set operations at all.
|
||||
|
||||
mightContain(x)
|
||||
expectedFalsePositiveProbab
|
||||
put
|
||||
approxElemCount() - estimates from fraction of set bits
|
||||
|
||||
Constructors - there should be these atleast
|
||||
(expectedPuts, numHashes, falsePositiveityRate)
|
||||
(bitSize, numHashes)
|
||||
|
||||
### The headers
|
||||
|
||||
using vector<bool> as it's bit packed internally and allows for
|
||||
runtime size declaration
|
||||
|
||||
I ended up writing the entire impl in the header file and then split it.
|
||||
Earlier I was going for a templated thing but that needed a lot of additional handling
|
||||
due to how something with internal pointers would manage data ( say std::string ).
|
||||
Now it's similar to xxhash where you pass in a void pointer and a length to it.
|
||||
|
||||
## Fixing clangd
|
||||
|
||||
Post the split, I needed to fix clangd as it did not
|
||||
|
||||
Reference in New Issue
Block a user