Comments on: Deduplicate any array in Javascript

By: John H

John H — Fri, 18 Dec 2009 14:43:56 +0000

Is it "just a little faster" because the variable, ary, is lower in the scope chain than array? Or is there some special magic here?

By: fearphage

fearphage — Sat, 12 Dec 2009 11:59:49 +0000

It can actually be just a little faster than that using the third parameter:

array.filter(function(item, i, ary) {
  return ary.indexOf(item) == i;
});

(Corrected the variable names)

By: fearphage

fearphage — Sat, 12 Dec 2009 11:58:40 +0000

It can actually be just a little faster than that using the third parameter::

array.filter(function(value, i, ary) {
  return ary.indexOf(item) == i;
});

By: fearphage

fearphage — Wed, 09 Dec 2009 17:05:13 +0000

Why not use the native filter for primitives?

array.filter(function(item, i) {
  return array.indexOf(item) == i;
});

I understand this won’t work for complex objects and if you want them sorted, you’ll have to do that afterwards.

By: Andrea Giammarchi

Andrea Giammarchi — Tue, 08 Dec 2009 23:58:39 +0000

indexOf is unfortunately expensive only in IE where it’s not native while splice could be expensive but above for loop is something extremely easy to maintain (5 line of code that should just work).
On the other hand I am using on up to two native methods against sort + forEach + map + push + all callbacks involved and executed for each index so I am kinda sure mine will perform 2 up to 5 times faster (in IE as example just sort is expensive).
Finally, if the goal is to make something portable for every case, objects included, with mine you are sure about unique primitives/instances so it depends what you need (if you need just to compare a single object property, mine won’t make sense)
Regards

By: John H

John H — Tue, 08 Dec 2009 22:40:02 +0000

Interesting! I wouldn’t have thought to use indexOf() or splice() since I just assumed those were expensive operations. I see you’ve minimized the impact of scanning the array by using the second argument for indexOf(). Clever. Some day I’ll do some real benchmarking to see what works best.

By: Andrea Giammarchi

Andrea Giammarchi — Tue, 08 Dec 2009 20:32:56 +0000

I know there is not compare function, but was not this good enough?

for(var i = 1, length = array.length, n; i < length; ++i){
    if(-1 < (n = array.indexOf(array[i - 1], i))){
        a.splice(n, 1);
        --length;
        --i;
    }
};

an optimized fast performances version could avoid the inline splice and store those index to discard (splice could cost a lot) in order to push only certain indexes (not duplicated)

By: Pankaj

Pankaj — Tue, 08 Dec 2009 11:22:36 +0000

Another great post as usual. Had checked out prototype.js sometime back but never dojo. Now that you mention it, will surely try my hands on it.

By: Tweets that mention Deduplicate any array in Javascript | Unscriptable.com -- Topsy.com

Tue, 08 Dec 2009 07:35:50 +0000

[...] This post was mentioned on Twitter by John Hann, jonathan 'j5' cook. jonathan 'j5' cook said: RT @unscriptable: Deduplicate any array in Javascript: http://bit.ly/7taOgm (Doesn’t everybody have to do this once in a while?) [...]