net.i2p.util
Class DecayingBloomFilter

java.lang.Object
  extended by net.i2p.util.DecayingBloomFilter
Direct Known Subclasses:
DecayingHashSet

public class DecayingBloomFilter
extends java.lang.Object

Series of bloom filters which decay over time, allowing their continual use for time sensitive data. This has a fixed size (currently 1MB per decay period, using two periods overall), allowing this to pump through hundreds of entries per second with virtually no false positive rate. Down the line, this may be refactored to allow tighter control of the size necessary for the contained bloom filters, but a fixed 2MB overhead isn't that bad. NOTE: At 1MBps, the tunnel IVV will see an unacceptable false positive rate of almost 0.1% with the current m and k values; however using DHS instead will use 30MB. Further analysis and tweaking for the tunnel IVV may be required.


Field Summary
protected  long _currentDuplicates
           
 
Constructor Summary
DecayingBloomFilter()
          noop for DHS
DecayingBloomFilter(I2PAppContext context, int durationMs, int entryBytes)
          Create a bloom filter that will decay its entries over time.
DecayingBloomFilter(I2PAppContext context, int durationMs, int entryBytes, java.lang.String name)
           
DecayingBloomFilter(I2PAppContext context, int durationMs, int entryBytes, java.lang.String name, int m)
           
 
Method Summary
 boolean add(byte[] entry)
          return true if the entry added is a duplicate
 boolean add(byte[] entry, int off, int len)
           
 boolean add(long entry)
          return true if the entry added is a duplicate.
 void clear()
           
 long getCurrentDuplicateCount()
           
 double getFalsePositiveRate()
           
 int getInsertedCount()
           
 boolean isKnown(long entry)
          return true if the entry is already known.
static void main(java.lang.String[] args)
          This filter is used only for participants and OBEPs, not IBGWs, so depending on your assumptions of avg. tunnel length, the performance is somewhat better than the gross share BW would indicate.
 void stopDecaying()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

_currentDuplicates

protected long _currentDuplicates
Constructor Detail

DecayingBloomFilter

public DecayingBloomFilter()
noop for DHS


DecayingBloomFilter

public DecayingBloomFilter(I2PAppContext context,
                           int durationMs,
                           int entryBytes)
Create a bloom filter that will decay its entries over time.

Parameters:
durationMs - entries last for at least this long, but no more than twice this long
entryBytes - how large are the entries to be added? if this is less than 32 bytes, the entries added will be expanded by concatenating their XORing against with sufficient random values.

DecayingBloomFilter

public DecayingBloomFilter(I2PAppContext context,
                           int durationMs,
                           int entryBytes,
                           java.lang.String name)
Parameters:
name - just for logging / debugging / stats

DecayingBloomFilter

public DecayingBloomFilter(I2PAppContext context,
                           int durationMs,
                           int entryBytes,
                           java.lang.String name,
                           int m)
Parameters:
m - filter size exponent
Method Detail

getCurrentDuplicateCount

public long getCurrentDuplicateCount()

getInsertedCount

public int getInsertedCount()

getFalsePositiveRate

public double getFalsePositiveRate()

add

public boolean add(byte[] entry)
return true if the entry added is a duplicate


add

public boolean add(byte[] entry,
                   int off,
                   int len)

add

public boolean add(long entry)
return true if the entry added is a duplicate. the number of low order bits used is determined by the entryBytes parameter used on creation of the filter.


isKnown

public boolean isKnown(long entry)
return true if the entry is already known. this does NOT add the entry however.


clear

public void clear()

stopDecaying

public void stopDecaying()

main

public static void main(java.lang.String[] args)
This filter is used only for participants and OBEPs, not IBGWs, so depending on your assumptions of avg. tunnel length, the performance is somewhat better than the gross share BW would indicate. Following stats for m=23, k=11: Theoretical false positive rate for 16 KBps: 1.17E-21 Theoretical false positive rate for 24 KBps: 9.81E-20 Theoretical false positive rate for 32 KBps: 2.24E-18 Theoretical false positive rate for 256 KBps: 7.45E-9 Theoretical false positive rate for 512 KBps: 5.32E-6 Theoretical false positive rate for 1024 KBps: 1.48E-3 Then it gets bad: 1280 .67%; 1536 2.0%; 1792 4.4%; 2048 8.2%. Following stats for m=24, k=10: 1280 4.5E-5; 1792 5.6E-4; 2048 0.14% Following stats for m=25, k=10: 1792 2.4E-6; 4096 0.14%