Commit Graph
129 Commits
Author SHA1 Message Date
Nicolas Williams 5bfb9781f7 Add Streaming parser (--stream)
Streaming means that outputs are produced as soon as possible.  With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.

This:

    $ jq -c --stream .

should produce the same output as this:

    $ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'

The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values.  The latter indicate that the array/object at
that path ended.

Scalars and empty arrays and objects are leaf values for this purpose.

For example, a truncated input produces a path as soon as possible, then
later the error:

    $ printf '[0,\n'|./jq -c --stream .
    [[0],0]
    parse error: Unfinished JSON term at EOF at line 3, column 0
    $
2014-12-26 23:05:56 -06:00
Nicolas Williams e51e916e31 Fix foreach non-progation of errors
Errors were being re-propagated as "break".

Also add `error/0` builtin, which made this fix easier.
2014-12-26 23:05:20 -06:00
Nicolas Williams c02fcc8fef Fix in and inside 2014-12-26 22:56:48 -06:00
Nico Williams 3234a7681e Merge pull request #366 from slapresta/inversed-contains
Added `in` command
2014-12-26 22:00:41 -06:00
Nico Williams 36077da66a Merge pull request #628 from slapresta/map-on-objects
Modify map\1 so that it works on objects
2014-12-26 21:50:11 -06:00
Nicolas Williams 0053aa868c Add @tsv; fix #645 2014-12-24 11:21:18 -06:00
Nico Williams 2775b90b20 Merge pull request #601 from slapresta/redefine-any-all
Define {any,all}/0,1 in terms of {any,all}/2
2014-12-23 23:37:25 -06:00
Nicolas Williams 9017b83d4e Make values faster (fix #652) 2014-12-23 23:22:57 -06:00
Santiago Lapresta 84d3203dd1 Define map_values 2014-12-22 23:05:35 +01:00
Santiago Lapresta a3234034b5 in is now inside, added in as inverse of has 2014-12-22 22:17:53 +01:00
Santiago Lapresta b748eae035 Added in command 2014-12-22 22:14:06 +01:00
Nicolas Williams be11b2768f Add until(cond; next); fix #639 2014-12-12 17:35:59 -06:00
William Langford 30e00820a7 Fix #600. Add regression test 2014-11-12 20:46:34 -05:00
Santiago Lapresta dc5e77ef8e Define {any,all}/2 independently from {any,all}/0 2014-10-21 13:00:38 +02:00
Santiago Lapresta 72735167f9 Define {any,all}/{0,1} in terms of {any,all}/2 2014-10-20 22:01:25 +02:00
William Langford 3411167c03 Properly call onig_error_code_to_str
Fix #585
2014-10-06 21:48:56 -04:00
pkoppstein 85f0e30c81 fix sub (#586); add gsub/3; add transpose/0.
Signed-off-by: William Langford <wlangfor@gmail.com>
2014-10-06 21:32:07 -04:00
Nicolas Williams 1796a716ea Restore split/1; don't use regexp (fix #576) 2014-10-03 17:58:45 -05:00
Nicolas Williams 0b29b44cab from-entries should work with EC2 (fix #592) 2014-10-03 17:47:13 -05:00
Nicolas Williams a0a350f468 Remove sort/1 and group/1
Reserve these for sorting with jq-coded comparators.
2014-10-03 17:47:13 -05:00
Nicolas Williams 4a57b84db0 to_entries should not sort keys (fix #561) 2014-09-30 21:52:30 -05:00
Nicolas Williams 9bd55487ea C-coded builtins should not assert (fix #553) 2014-08-21 00:12:29 -05:00
Nicolas Williams 1ba8c2cfa6 Add module directive, modulemeta builtin
Fix #425.
2014-08-14 03:26:26 -05:00
Nicolas Williams 908c10928b Fix #549 2014-08-11 14:58:44 -05:00
Nicolas Williams 8a561120c4 More constant folding: null, true, and false
A step towards doing constant folding for arrays and objects.
2014-08-09 19:15:50 -05:00
Nicolas Williams 4249bbf0d5 Make nwise private 2014-08-08 19:36:43 -05:00
Nicolas Williams 8a57c27e92 Make regexp builtins and range/3 use #524 too 2014-08-08 19:31:11 -05:00
Nicolas Williams 8706b6b16a Use def f($a): ...; syntax for builtins 2014-08-08 19:23:46 -05:00
pkoppstein a696c6b551 regex filters (#432): scan, splits, split, sub, gsub 2014-08-08 17:00:14 -05:00
pkoppstein f8fed0f0bc Apply TCO to recurse/1, add recurse/2; tweak docs
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-08-04 21:11:01 -05:00
Nicolas Williams a2f99d43c3 Some builtins are unnecessarily special (fix #521) 2014-08-01 00:20:30 -05:00
pkoppstein 53993a9942 Add capture; document regular expression filters
Fix #493.
2014-07-28 19:41:21 -05:00
Nicolas Williams e1b20b48f0 Add first, nth, last (fix #510) 2014-07-28 13:18:58 -05:00
William Langford 38b939688a Added library system with -l, -L, and JQ_LIBRARY_PATH
Created util.[ch] to hold common utilities.
2014-07-22 22:51:11 -05:00
Nicolas Williams 10477fc2d4 Make while() handle break 2014-07-09 00:39:25 -05:00
Nicolas Williams d8672e1b2a Make C-coded built-ins take jq_state * argument 2014-07-07 23:37:42 -05:00
Nicolas Williams a68958e5dc error(x) should not tostring its arg; fix #466 2014-07-07 22:26:53 -05:00
Nicolas Williams 824f7df404 limit should use break 2014-07-07 22:26:07 -05:00
Nicolas Williams 4bcff40828 Make any/2 and all/2 efficient using foreach 2014-07-07 22:25:34 -05:00
Nicolas Williams 539dccae90 Add break builtin for foreach 2014-07-07 19:33:17 -05:00
Nicolas Williams 821cd31e67 Add any/N and all/N x N in (1, 2) (fix #455)
Contributed by @pkoppstein.
2014-07-06 20:08:06 -05:00
Nicolas Williams 5a863bf010 Add foreach and limit 2014-07-06 19:40:05 -05:00
Nicolas Williams 78a8419428 Add while(cond; update) (fix #314) 2014-07-02 22:05:33 -05:00
Nicolas Williams ff9a61ead2 Add range(init;upto;by) (fix #317) 2014-07-02 22:05:33 -05:00
William Langford 9f4d7e2fb8 Fixed base64 issue with UTF-8 strings
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-01 22:56:13 -05:00
Zhiming Wang a5e9086676 Let @uri produce uppercase hexadecimal digits...
instead of lowercase ones.

According to RFC 3986,

   The uppercase hexadecimal digits 'A' through 'F' are equivalent to
   the lowercase digits 'a' through 'f', respectively.  If two URIs
   differ only in the case of hexadecimal digits used in percent-encoded
   octets, they are equivalent.  For consistency, URI producers and
   normalizers should use uppercase hexadecimal digits for all percent-
   encodings.

See https://github.com/stedolan/jq/issues/451 for details.

Test suite and manual are also updated to reflect this change.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-30 19:52:42 -05:00
Nicolas Williams ff5cf0abc7 Remove extra free of testmode
I added a jv_free() too many while rebasing @wtlangford's contribution.
It could only be triggered by calling `_match_impl` directly.
2014-06-18 21:11:06 -05:00
William Langford 8ff935c01a Added regex support as per issue #164.
jq now depends on oniguruma for regex support.
Modified configure.ac accordingly.

Added valgrind suppression file for oniguruma to prevent one-time and bounded
leaks from causing tests to fail.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-18 21:02:47 -05:00
Santiago Lapresta dde43f796e Simplified standard library
Close #426.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-17 18:17:50 -05:00
Nicolas Williams f58f771660 Fix #280: from_entries of [] is null, should be {} 2014-06-16 22:40:44 -05:00