array filter?
brentp opened this issue · comments
cool project, I was looking at the tests and saw this:
let aArray = [2, 8, -4]
test "array filter":
check((aArray --> filter(it > 2)) == [8, 0, 0])
I would expect [8]
Yes, I think the real use-case for array-filter is not yet clear. Maybe it should be prevented. Especially since the array size is only known at compile-time and the filter operation is performed at run-time, the output can not be shrinked.
The array was introduced to perform in-place operations, which makes sense for map and the other operations which return an integral result.
Yes, I am going to remove filter for now, as it can't be followed by other operations correctly.
I've thought of two possible solutions:
aArray --> filterCount(it > 2) == ([8, 0, 0], 1)or
aArray -> filter(it > 2) == arraySlice([8, 0, 0], 1) # [8]where
type
ArraySlice*[T] = object
arr*: ptr T
first*: uint
last*: uintor similar. Array slices make more sense if one wants to apply several operations in place including filter. For new arrays filterCount is the most reasonable options
I really don't know - thing is, that up to now the macro code itself was - at least kind of - straight forward. I'm not sure anyone will need the array filter - at least not as a result. Clearly when the operation is finished with a single type chaining with all, fold, etc. we don't have a problem here. But I can't think of a language that would support an array.filter in that way.
Just my opinion: but how about when filter is used and the final result is an iterable type, the final type should just be a seq!
.. on the other hand the result ([8, 0, 0], 1) could be called with toSeq and we have a sequence...
But: that behavior gets kind of erratic - I'd prefer an error message
'filtered array is not supported as a result type - use a seq instead'
Maybe the user could use the foreach instead and apply the changes on the array him/herself.
I think returning a seq would be surprising.
I plan on removing filter for array and adding filterToSeq only for array, so people don't get a seq by mistake
I agree, filter should just return a seq doing something else would had more cognitive overload "Oh what was zero_functional doing for filter for arrays again, have to read the docs".
The only language we can compare to is Rust, others like Ocaml, Haskell, Python ... uses Linked List so size is not known at run time.
Rust answer is simple: there is no FromIterator traits that allows collection of iterator chaining into an array: https://stackoverflow.com/questions/26757355/how-do-i-collect-into-an-array
To allow for less disjointed interface, maybe zip, map, filter etc should always return a seq, and we can have a ForEach macro that takes a varargs of arrays?
I still think supporting operations resulting in arrays (or even other types of collections) is useful.
I've missed dictionary and set comprehensions from Python, similar idioms are also important in Ruby.
We can also expand the notation
aArray => map(..) =>@ filter(..) # map returns an array, filter returns a seq
aTable => map(..) =>@ filter(..) => all(..) # map returns table, after that we work with a seqThis way => will result in <inputType> = <outputType> and =>@ in <outputType> = seq[args of input]
Nice use of nim flexibility!
(It can be also combined with defaulting on @michael72 's => instead of -->: --> can be deprecated but still left in for backwards compat).
I'll implement that in the evening if it seems suitable
differentiate between =>@ and => is actually a great idea! - I hope it won't be a hell to implement ;-)
Darn - I just saw, that => is already part of the future module ... (forgot about that, although I've used it already)
so....
-->@ (looks a bit like a rose ;) <--@)
What about
==> and =>@ =>= and =>@
or maybe
|=> and |=>@ hm....
|-> and |->@
-- could be named as "pipeInto" or "pipeIntoSeq"
hm... |> already exists in some other languages meaning: use the result of the left side and apply it to the right side...
myList --> map(it + 1) myList -->@ filter(it > 0)
myList ->= map(it + 1) myList ->@ filter(it > 0)
myList ==> map(it + 1) myList =>@ filter(it > 0)
myList =>= map(it + 1) myList =>@ filter(it > 0)
myList |=> map(it + 1) myList |=>@ filter(it > 0)
myList |-> map(it + 1) myList |->@ filter(it > 0)I think I'm fine with anything :)
Great analysis, => is clashing with future indeed. I like the ->= and ->@ idea, it makes sense, but it seems pretty hard to remember: 3 different sigils, so continuing with the current notation might be best.
--> and -->@ ?
3 (or 4) different sigils might be too much maybe - yeah
OK then
myList --> map(it + 1)
myArray -->@ filter(it > 0)alternatively maybe
myList ==> map(it + 1)
myArray ==>@ filter(it > 0)looks alright ;-)
yeah - stick with the `-->´ then :+1
Now I remember: the dsl notation is based on a single operator. That makes it easy to analyze the whole chain as . has higher precedence
base --> a.b.c also
base --> zipWith(other).map(f)# -->
base <optimize those operations on it> a.b Now, a --> b --> c should be equivalent, but slower as you can benchmark. Maybe there is a way to make it work like this but I am not sure how without an additional element
zero:
a --> b --> cSo, my new idea is to
a --> map().mapIndexed().filter@() # <handler>@ returns always a seq, <handler> returns InputType for map, filter If you can think of a way to make the other syntax work (because now it's (a --> b) --> c: it's expanded into two loops with new seq-s), go ahead !
Why not the second solution - before turning it all upside down.
You could also call it filterSeq btw. Then working on seq.filterSeq has no real benefit - but it would return a seq in case an array is supplied and it is the last argument (or the last before map).
Should be easy to implement (as you can easily get the type of an element - see my find implementation)
Maybe we could use a branch for developing new features btw - maybe even one branch for each feature
Yeah, filter@ is basically a little bit shorter syntax for filterSeq. Maybe <handler>Seq is a bit more reasonable, I'll think about it.
That's true, I'll work in a branch
@brentp
Back to the original issue - I think we can close this now.
The test code - in case of array now would look like:
let aArray = [2, 8, -4]
test "array filter":
check((aArray --> filter(it > 2)) == [0, 8, 0])
test "array filterSeq":
check((aArray --> filterSeq(it > 2)) == @[8])This is "nulling" of all entries in the matrix where the given condition is false in the case with the array. With filterSeq the size of the resulting sequence is reduced.
So the benefit is still questionable in case of filter on array, but it could potentially be used to 0 vector entries or matrix entries (depending on what the array shall represent).
It is not possible to produce a smaller array as output at runtime.
cheers. I'll git it a try.