perf: avoid the intermediate array in from_entries (#3626)
Define `from_entries` as `add(.[] | f) // {}` instead of
`map(f) | add // {}`. Since `add(f)` is `reduce f as $x (null; . + $x)`,
both fold the same values in the same order from the same seed, so the
only difference is that the array of single-key objects built by `map`
is never materialized.
Measured with both the generator and the result suppressed, so that only
the `from_entries` work differs: 6-15% less user time across input shapes,
which is 10-24% of what `from_entries` itself costs. For inputs above
roughly a thousand entries, peak RSS drops by 37%, around 80% of the
memory `from_entries` adds on top of its input; a 1,000,000 entry array
goes from 1064 MiB to 672 MiB.
This commit is contained in:
1 parent
12cdd950e9
commit
4beed4e01a
1 file changed
+2
-2
+2
-2
@@ -39,8 +39,8 @@ def recurse(f; cond): def r: ., (f | select(cond) | r); r;
|
||||
def recurse: recurse(.[]?);
|
||||
|
||||
def to_entries: [keys_unsorted[] as $k | {key: $k, value: .[$k]}];
|
||||
def from_entries: map({ (.key // .Key // .name // .Name):
|
||||
if has("value") then .value else .Value end }) | add // {};
|
||||
def from_entries: add(.[] | { (.key // .Key // .name // .Name):
|
||||
if has("value") then .value else .Value end }) // {};
|
||||
def with_entries(f): to_entries | map(f) | from_entries;
|
||||
def reverse: [.[range(length-1;-1;-1)]];
|
||||
def indices($i): if type == "array" and ($i|type) == "array" then .[$i]
|
||||
|
||||
Reference in new issue
Block a user