Commit Graph
100 Commits
Author SHA1 Message Date
Nicolas Williams 862696a778 Add missing test file 2015-01-01 01:39:50 -06:00
Nicolas Williams 86bc662bd2 Move some module tests into all.test 2014-12-31 20:09:53 -06:00
Nicolas Williams ae7f8d6ab9 Further module system revamp (fix #659)
To import a module now use:

    # Import module.jq file:
    import "relative/path/to/module" as foo;

    # Use the module's defs as foo::<def-name>

To import a JSON file:

    # Read file.json:
    import "relative/path/to/file" as $foo;
    #
    # Use as $foo::foo

Using `-L` now drops the builtin library path and appends the requested
path to the empty array (or the result of an earlier `-L`).

Support for the `$JQ_LIBRARY_PATH` environment variable has been
removed.
2014-12-31 20:09:53 -06:00
Nicolas Williams 7dc34b92af Add label $name | EXP; fix break
This is to fix the problem where `break` is dynamic, not lexical.

With this it should be possible to do this sort of thing:

    label $break | inputs | if ... then $break|error else . end

This is a backwards-incompatible change for master, but the previous
`break` hadn't shipped yet.

Still needed:

 - testing
2014-12-30 11:42:45 -06:00
Nicolas Williams cbfc0d6130 Remove string indexing by string (fix #454)
This turns out to have been a bad idea:

    "foo"|.["o"]

it interacts badly with `path()`.

See #454 for the gory details.
2014-12-30 11:31:52 -06:00
Nicolas Williams c308b2881f Fix parser leak (fuzzing) 2014-12-30 11:31:52 -06:00
Nicolas Williams c959d1fca1 Fix lineno counting in jq_test.c 2014-12-30 11:28:50 -06:00
Nicolas Williams 63511a874f Add support for testing erroneous programs
This will allow moving tests of some tests from tests/run to
tests/all.test.  It will also be useful for more extensive testing of
language features where we want nice error messages, or errors at all,
to be produced, particularly language features where the errors arise at
compile-, codegen-, or link-time rather than parse-time.

This does not check for run-time errors though -- those we can already
check for with try/catch.

    $ jq --run-tests
    %%FAIL
    break
    jq: error: break used outside labeled control structure
    ^D
    1 of 1 tests passed (0 malformed)
    $
2014-12-30 10:55:30 -06:00
Nicolas Williams 90c705f138 Make --run-tests more informative
Print the line number and program text on the same output line where a
failure is reported, for all failures that were already reported on
lines starting with '***'.  This makes poring over test logs with
failures much easier.
2014-12-30 10:55:30 -06:00
Nicolas Williams b67bad82cb Allow resetting of jq err callback
This will be useful for the upcoming test-erroneous-programs improvement
to --run-tests, so we can switch between the default error reporting
method (print to stderr) to a method internal to --run-tests, and back.

The idea is that when testing programs that are expected to compile (and
link), it'd be nice if errors continue going to stderr, while when
testing programs that must fail to compile (or link), the error has to
be captured so it can be compared to the error expected by the test.
2014-12-30 10:52:38 -06:00
Nicolas Williams 6981b46338 Fix streaming bug (fuzzing) 2014-12-30 01:35:22 -06:00
Nicolas Williams 5df20f4954 Add debug builtin
And refactor setup of jv dump flags.
2014-12-27 18:25:34 -06:00
Nicolas Williams a9c613e87d Don't force C API users to set input cb 2014-12-27 18:15:10 -06:00
Nicolas Williams 7538ef34be Make jq --run-tests show test line numbers 2014-12-27 18:15:10 -06:00
Nicolas Williams 5c62a4b805 Fix uninitialized struct field 2014-12-27 18:09:28 -06:00
Nicolas Williams 559863ca68 Streaming parser torture tests 2014-12-26 23:18:07 -06:00
Nicolas Williams 845ad5e969 Fuzz JSON parser 2014-12-26 23:05:57 -06:00
Nicolas Williams 243b1f8e5e Better quoting in tests/run 2014-12-26 23:05:56 -06:00
Nicolas Williams 5bfb9781f7 Add Streaming parser (--stream)
Streaming means that outputs are produced as soon as possible.  With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.

This:

    $ jq -c --stream .

should produce the same output as this:

    $ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'

The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values.  The latter indicate that the array/object at
that path ended.

Scalars and empty arrays and objects are leaf values for this purpose.

For example, a truncated input produces a path as soon as possible, then
later the error:

    $ printf '[0,\n'|./jq -c --stream .
    [[0],0]
    parse error: Unfinished JSON term at EOF at line 3, column 0
    $
2014-12-26 23:05:56 -06:00
Nicolas Williams 906d2537b9 Allow C-coded functions to empty
Just return a jv_invalid() without a message.
2014-12-26 23:05:56 -06:00
Nicolas Williams bbb287f54b Add BLOCK_8() macro 2014-12-26 23:05:56 -06:00
Nicolas Williams e51e916e31 Fix foreach non-progation of errors
Errors were being re-propagated as "break".

Also add `error/0` builtin, which made this fix easier.
2014-12-26 23:05:20 -06:00
Nicolas Williams 46e3125e26 Allow zero-length buffers in jv_parser_set_buf()
If the caller is at EOF and has no more bytes to feed the parser, how is
the parser to be told about the EOF condition?  A small fix to allow
zero-length buffers in jv_parser_set_buf() fixes this problem (it also
makes it easier to deal with async I/O: feed the parser whatever is
available, including nothing).
2014-12-26 22:56:48 -06:00
Nicolas Williams c02fcc8fef Fix in and inside 2014-12-26 22:56:48 -06:00
Nico Williams 25d4cbd1b3 Merge pull request #624 from Janrain-Colin/patch-1
Doc correction
2014-12-26 22:14:17 -06:00
Nico Williams 3234a7681e Merge pull request #366 from slapresta/inversed-contains
Added `in` command
2014-12-26 22:00:41 -06:00
Nico Williams 36077da66a Merge pull request #628 from slapresta/map-on-objects
Modify map\1 so that it works on objects
2014-12-26 21:50:11 -06:00
Nicolas Williams 9e1fd8cf4d Fix infinite loop in read_more() (fix #656) 2014-12-24 16:49:55 -06:00
Nicolas Williams a55085b662 Fix EOF handling; fix #656 2014-12-24 16:07:36 -06:00
Nico Williams e0f9b6a5cd Merge pull request #651 from eiiches/fix-examples
Fix several errors in the manual
2014-12-24 11:29:29 -06:00
Nicolas Williams 0053aa868c Add @tsv; fix #645 2014-12-24 11:21:18 -06:00
Nico Williams fab2048648 Merge pull request #653 from msabramo/patch-1
.travis.yml: Set sudo false; use containers
2014-12-24 11:20:47 -06:00
Nicolas Williams 56ae88d9d5 Module search revamp for pkg managers
The search path listed in an import directive can now be an array.  The
top-level search path is appended.  Null and empty strings in the path
terminate any search.  The "." in "." and "./*" is replaced with the
directory containing the file doing the import (for command-line
programs this is the current directory, though that may be a bad idea).

No version numbers or anything of the sort are gratuitously added to the
search paths.

All this makes external package managers possible by allowing
dependencies to be installed local to dependents.
2014-12-24 02:31:51 -06:00
Nicolas Williams 1cdcff1c01 Fix bug caused by 8ca07a0 2014-12-24 02:25:11 -06:00
Nicolas Williams 8ca07a08c3 Fix #348: reject unescaped control chars 2014-12-24 00:01:43 -06:00
Nico Williams 2775b90b20 Merge pull request #601 from slapresta/redefine-any-all
Define {any,all}/0,1 in terms of {any,all}/2
2014-12-23 23:37:25 -06:00
Nicolas Williams ae312bd7fe Use __attribute__ __printf__ with GCC 2014-12-23 23:22:57 -06:00
Nicolas Williams 9017b83d4e Make values faster (fix #652) 2014-12-23 23:22:57 -06:00
Nicolas Williams 02cf1831e9 Fix #649 2014-12-23 18:16:21 -06:00
Nico Williams 54a0470c42 Merge pull request #611 from StevenMaude/tutorial-typo-fixes
Fix typos in tutorial
2014-12-12 17:46:17 -06:00
Nico Williams 3dd790c408 Merge pull request #616 from ianmiell/ianmiell_requirements
Requirements made slightly more complete
2014-12-12 17:45:17 -06:00
Nicolas Williams be11b2768f Add until(cond; next); fix #639 2014-12-12 17:35:59 -06:00
Nicolas Williams 2075bec60f Fix #647, but caused by 89791a0 2014-12-12 17:05:16 -06:00
Nicolas Williams 44c2382402 Add --argjson, fix #648 2014-12-12 16:40:07 -06:00
Nico Williams fd041c2064 Merge pull request #642 from Janrain-Colin/patch-3
fix typo in manual
2014-12-12 16:03:18 -06:00
Nicolas Williams 7243989c52 Fix refcount leak, fix #618 2014-11-29 15:24:43 -06:00
Nicolas Williams 46613af64c STOREV/LOADV* should also print refcnts 2014-11-28 20:52:22 -06:00
Nicolas Williams 79000644c6 Enable printing of stack val refcnts 2014-11-28 19:35:29 -06:00
Nicolas Williams b349e7d9a8 Print stack value refcounts when tracing (#636) 2014-11-28 19:26:41 -06:00
Nicolas Williams 89791a000b Add support for JSON sequence MIME type
Per draft-ietf-json-text-sequence-07 (which soon will be published as an
RFC).
2014-10-12 08:44:40 -05:00
Nicolas Williams 1796a716ea Restore split/1; don't use regexp (fix #576) 2014-10-03 17:58:45 -05:00
Nicolas Williams 07d8d5c5ef Update docs about sort/group/min/max/unique 2014-10-03 17:58:44 -05:00
Nicolas Williams 0b29b44cab from-entries should work with EC2 (fix #592) 2014-10-03 17:47:13 -05:00
Nicolas Williams a0a350f468 Remove sort/1 and group/1
Reserve these for sorting with jq-coded comparators.
2014-10-03 17:47:13 -05:00
Nicolas Williams 4a57b84db0 to_entries should not sort keys (fix #561) 2014-09-30 21:52:30 -05:00
Nicolas Williams 456bafa82f Drop the jq version directory from search path 2014-08-30 00:58:24 -05:00
Nicolas Williams 7a8e3c759e Never close stdin; allow multiple - arguments 2014-08-30 00:40:03 -05:00
Nicolas Williams b70bea8d34 Handle invalid inputs in argument files (fix #562) 2014-08-30 00:40:03 -05:00
Nicolas Williams 9bd55487ea C-coded builtins should not assert (fix #553) 2014-08-21 00:12:29 -05:00
Nicolas Williams 23e2e2eab3 Quiet warning about freopen() of stdout 2014-08-20 20:49:30 -05:00
Nicolas Williams 8d2d5e37e5 Drop "any/" in module search; use 1.x-master 2014-08-20 20:48:48 -05:00
Nicolas Williams 287e2c96ed BSD sed lacks -r; fix #551 2014-08-14 03:31:25 -05:00
Nicolas Williams f8d9478517 Document module system
Fix #511.
2014-08-14 03:26:28 -05:00
Nicolas Williams 1ba8c2cfa6 Add module directive, modulemeta builtin
Fix #425.
2014-08-14 03:26:26 -05:00
Nicolas Williams d654724fef Add JV_OBJECT() macro 2014-08-14 03:21:38 -05:00
Nicolas Williams 3375089013 Add JV_ARRAY() macro 2014-08-14 03:21:38 -05:00
Nicolas Williams 9f13afa20f Add jq_report_error() function; use it
Put a stop to fprintf(stderr, ...) where we shouldn't.
2014-08-14 03:21:35 -05:00
Nicolas Williams 908c10928b Fix #549 2014-08-11 14:58:44 -05:00
Nicolas Williams c321c3b86b Constant fold objects 2014-08-09 20:47:03 -05:00
Nicolas Williams f87521183f Fold constant arrays 2014-08-09 20:27:08 -05:00
Nicolas Williams 8a561120c4 More constant folding: null, true, and false
A step towards doing constant folding for arrays and objects.
2014-08-09 19:15:50 -05:00
Nicolas Williams 20ca21cb0d Out of bounds negative array indices should raise 2014-08-09 19:15:50 -05:00
Nicolas Williams 2d1a75f550 .foo[-1] = ... trips assertion (fix #490) 2014-08-09 13:10:41 -05:00
Nicolas Williams 8cddb7c681 Allow any number of jq-coded function arguments 2014-08-09 12:42:39 -05:00
Nicolas Williams 4249bbf0d5 Make nwise private 2014-08-08 19:36:43 -05:00
Nicolas Williams 8a57c27e92 Make regexp builtins and range/3 use #524 too 2014-08-08 19:31:11 -05:00
Nicolas Williams 8706b6b16a Use def f($a): ...; syntax for builtins 2014-08-08 19:23:46 -05:00
Nicolas Williams 32c08d48f3 Add def f($arg): syntax (fix #524) 2014-08-08 19:16:01 -05:00
Nicolas Williams fc52e78ed4 Fix tests for #432 2014-08-08 18:17:58 -05:00
Nicolas Williams eb12f0358e Fix docs YAML (introduced in #432) 2014-08-08 18:01:42 -05:00
Nicolas Williams 0d437e25de Better error msg for bad shell quoting (fix #538) 2014-08-06 17:51:53 -05:00
Nicolas Williams 2159f9f5e0 Document TCO a bit more 2014-08-05 01:15:33 -05:00
Nicolas Williams 2518b3a34f Fix docs (missing backquotes) 2014-08-05 01:15:33 -05:00
Nicolas Williams a2f99d43c3 Some builtins are unnecessarily special (fix #521) 2014-08-01 00:20:30 -05:00
Nicolas Williams a7878a8d1a Correct @wtlangford's address 2014-07-30 13:19:46 -05:00
Nicolas Williams c73ad74be0 Add @wtlangford to the AUTHORS list 2014-07-30 13:02:34 -05:00
Nicolas Williams e1b20b48f0 Add first, nth, last (fix #510) 2014-07-28 13:18:58 -05:00
Nicolas Williams 2e2538ccb8 Fold constants (fix #504) 2014-07-27 17:48:49 -05:00
Nicolas Williams ae27178352 Fix #499, #500 2014-07-27 17:41:40 -05:00
Nicolas Williams 10f5a3448d Do fix remaining bison warning (fix #433) 2014-07-22 22:51:11 -05:00
Nicolas Williams 50b85c2911 Rename jq_memmem() to _jq_memmem() 2014-07-22 22:51:11 -05:00
Nicolas Williams 01fc8168e9 Add -i option to edit files in place (fix #105) 2014-07-20 00:11:23 -05:00
Nicolas Williams 3362fb3406 Add -n to short usage msg; fix --arg msg 2014-07-20 00:04:24 -05:00
Nicolas Williams 8c3d503d54 Fix bison empty rule warnings (#433, partial fix) 2014-07-14 16:41:48 -05:00
Nicolas Williams 2bb9fc5dda Fix #484, try/catch syntax has conflicts 2014-07-14 16:41:48 -05:00
Nico Williams ea3e947b50 Merge pull request #483 from else/master
jq 1.4 is in Debian
2014-07-14 09:16:01 -05:00
Nico Williams 918ada06dc Merge pull request #482 from mbruggmann/manual-fixes
Fix manual example for `endswith`.
2014-07-13 19:01:57 -05:00
Nicolas Williams cb2737a433 Add regression test for #479 2014-07-13 19:00:37 -05:00
Nicolas Williams c38e5b1b2c Lib defs don't bind to each other; fix #479 again 2014-07-13 18:36:27 -05:00
Nicolas Williams cadf8c3cb0 Revert "Library defs don't bind to each other; fix #479"
This reverts commit 6b6e3f4262.  It was
the wrong fix.
2014-07-13 02:04:27 -05:00