bloom filter , what wrong ? why false ?

Question

bloom filter , what wrong ? why false ?

paulanalyst opened this issue 10 years ago · comments

_ _ ()_ | A fresh approach to technical computing
() | () () | Documentation: http://docs.julialang.org
_ _ | | __ _ | Type "help()" to list help topics
| | | | | | |/ ` | |
| | || | | | (| | | Version 0.3.0-prerelease+2599 (2014-04-11 23:52 UTC)
/ |_'|||__'| | Commit bf7096c (51 days old master)
|__/ | x86_64-w64-mingw32

julia> using BloomFilters

julia> n, k = 100, 50
(100,50)

julia> ety=readcsv("etykiety_kli.txt")
56522x1 Array{Any,2}:
"EC00113876"
"EC00085985"
"EC00037297"
"EC00005413"
"EC00126328"
"EC00021867"
"EC00114062"
"EC00007751"
"EC00206892"
"EC00115609"
?
"EC00159409"
"EC00172340"
"EC00062096"
"EC00134183"
"EC00108009"
"EC00050665"
"EC00081817"
"EC00155357"
"EC00031904"
"EC00060934"

julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 100
Number of Hashes: 50

julia> add!(filter, ety)

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
false

Why false ?

julia> findfirst(ety,x)
2

Paul

John Myles White · Answer 1 · Mon Jun 02 2014 22:58:08 GMT+0800 (China Standard Time)

What's the SHA1 of the version of this package you're using?

paulanalyst · Answer 2 · Tue Jun 03 2014 00:29:44 GMT+0800 (China Standard Time)

BloomFilters,0.0.0

John Myles White · Answer 3 · Tue Jun 03 2014 00:33:22 GMT+0800 (China Standard Time)

Ok. I think we may need to change the released version.

paulanalyst · Answer 4 · Tue Jun 03 2014 00:43:31 GMT+0800 (China Standard Time)

I downloaded today.

John Myles White · Answer 5 · Tue Jun 03 2014 01:06:01 GMT+0800 (China Standard Time)

Yes, the released version is not up-to-date.

John Myles White · Answer 6 · Tue Jun 03 2014 01:17:12 GMT+0800 (China Standard Time)

Try updating now. You need Julia 0.3-.

paulanalyst · Answer 7 · Tue Jun 03 2014 01:30:51 GMT+0800 (China Standard Time)

Ok, updated, BloomFilters,0.0.1

but :
julia> using BloomFilters

julia> n, k = 100, 50
(100,50)

julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 100
Number of Hashes: 50

julia> ety=readcsv("etykiety_kli.txt")
56522x1 Array{Any,2}:
"EC00113876"
"EC00085985"
"EC00037297"
"EC00005413"
"EC00126328"
"EC00021867"
"EC00114062"
"EC00007751"
"EC00206892"
"EC00115609"
?
"EC00159409"
"EC00172340"
"EC00062096"
"EC00134183"
"EC00108009"
"EC00050665"
"EC00081817"
"EC00155357"
"EC00031904"
"EC00060934"

julia> add!(filter, ety)

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
false

julia> findfirst(ety,x)
2

paulanalyst · Answer 8 · Tue Jun 03 2014 01:33:28 GMT+0800 (China Standard Time)

ety is Array{Any,2}: ...

julia> add!(filter, ety)

why :
julia> contains(filter,ety)
true
???

julia> filter
A Bloom filter

Mask Size: 100
Number of Hashes: 50

julia> ety
56522x1 Array{Any,2}:
"EC00113876"
"EC00085985"
"EC00037297"
"EC00005413"
"EC00126328"
"EC00021867"
"EC00114062"
"EC00007751"
"EC00206892"
"EC00115609"
?
"EC00159409"
"EC00172340"
"EC00062096"
"EC00134183"
"EC00108009"
"EC00050665"
"EC00081817"
"EC00155357"
"EC00031904"
"EC00060934"

julia> contains(filter,vec(ety))
56522-element BitArray{1}:
true
true
true
true
true
true
true
true
true
true
?
true
true
true
true
true
true
true
true
true
true

julia>

paulanalyst · Answer 9 · Tue Jun 03 2014 01:35:08 GMT+0800 (China Standard Time)

after vec
julia> contains(filter,"somethink")
true
:/

John Myles White · Answer 10 · Tue Jun 03 2014 01:51:17 GMT+0800 (China Standard Time)

Ok. I'll look into this. It might take me a couple of weeks.

paulanalyst · Answer 11 · Tue Jun 03 2014 01:57:26 GMT+0800 (China Standard Time)

is better if data is no to long add!(filter,ety[1:50]) and when n is hi +-10000:
julia> using BloomFilters

julia> n, k = 100, 50
(100,50)

julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 100
Number of Hashes: 50

julia> ety=readcsv("etykiety_kli.txt")
56522x1 Array{Any,2}:
"EC00113876"
"EC00085985"
"EC00037297"
"EC00005413"
"EC00126328"
"EC00021867"
"EC00114062"
"EC00007751"
"EC00206892"
"EC00115609"
?
"EC00159409"
"EC00172340"
"EC00062096"
"EC00134183"
"EC00108009"
"EC00050665"
"EC00081817"
"EC00155357"
"EC00031904"
"EC00060934"

julia> add!(filter,ety[1:50])

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
true

julia> findfirst(ety,x)
2

julia> contains(filter,ety[1])
true

julia> contains(filter,ety[50])
true

julia> contains(filter,ety[51])
true

julia> contains(filter,ety[55])
false

julia> contains(filter,ety[505])
true

julia> using BloomFilters

julia> n, k = 10000, 50
(10000,50)

julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 10000
Number of Hashes: 50

julia> ety=readcsv("etykiety_kli.txt")
56522x1 Array{Any,2}:
"EC00113876"
"EC00085985"
"EC00037297"
"EC00005413"
"EC00126328"
"EC00021867"
"EC00114062"
"EC00007751"
"EC00206892"
"EC00115609"
?
"EC00159409"
"EC00172340"
"EC00062096"
"EC00134183"
"EC00108009"
"EC00050665"
"EC00081817"
"EC00155357"
"EC00031904"
"EC00060934"

julia> add!(filter,ety[1:50])

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
true

julia> findfirst(ety,x)
2

julia> contains(filter,ety[505])
false

julia> contains(filter,ety[50])
true

julia> contains(filter,ety[51])
false

julia> contains(filter,ety[52])
false

julia> contains(filter,ety[53])
false

julia> contains(filter,ety[55])
false

julia> contains(filter,ety[49])
true

julia> contains(filter,ety[48])
true

julia> contains(filter,ety[47])
true

julia>

paulanalyst · Answer 12 · Tue Jun 03 2014 02:17:56 GMT+0800 (China Standard Time)

must be ety[:]
OLD:
julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 1000
Number of Hashes: 50

julia> add!(filter,ety)

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
false

NEW:
julia> filter = BloomFilter(n, k)
A Bloom filter

Mask Size: 1000
Number of Hashes: 50

julia> add!(filter,ety[:])

julia> x=ety[2]
"EC00085985"

julia> contains(filter,x)
true

John Myles White · Answer 13 · Sun Dec 07 2014 02:53:36 GMT+0800 (China Standard Time)

Try again with 0.1.0