Commit Graph
100 Commits
Author SHA1 Message Date
Nicolas Williams 7fd9e86ea6 Make limit/2 more efficient
Contributed by @pkoppstein.
2018-03-06 16:35:33 -06:00
Nicolas Williams e113c67d51 limit/2 evals exp one time too many 2018-03-06 11:29:13 -06:00
Nicolas Williams 8eff744eec Update AUTHORS 2017-12-11 12:22:02 -06:00
Nicolas Williams 9a4576e756 Revert "reduce: handle empty updates (fix #1313)"
This reverts commit e24af3c78e.

While the semantics are desirable, there is no way to implement them
efficiently.  The reason is that in order to handle backtracking (empty)
from the state update expression, we have to retain a reference to the
reduction state value in order to restore it upon backtracking.
Retaining a reference to the reduction state kills performance by
causing lots of additional memory allocations and garbage because the
input to the update expression will always have at least two references,
thus no changes to it can be done in-place, and all changes end up being
CoW changes.

Avoiding this is the very reason for the LOADVN instruction (leaving
`null` in the variable loaded from).
2017-12-11 11:20:16 -06:00
Nicolas Williams b5560d8420 Fix import as $data linking 2017-12-10 23:09:35 -06:00
Nicolas Williams 18879fb993 Fix leak on error in strptime 2017-11-23 01:51:22 -06:00
Nicolas Williams c538237f4e Deal with strptime() on OS X and *BSD (fix #1415)
strptime() on OS X and *BSDs (reputedly) does not set tm_wday and
tm_yday unless corresponding %U and %j format specifiers were used.
That can be... surprising when one parsed year, month, and day anyways.
Glibc's strptime() conveniently sets tm_wday and tm_yday in those cases,
but OS X's does not, ignoring them completely.

This commit makes jq compute those where possible, though the day of
week computation may be wrong for dates before 1900-03-01 or after
2099-12-31.
2017-05-21 16:24:48 -05:00
Nicolas Williams 4a6241be06 Attempt to fix #1415
OS X (and *BSD) strptime() does not set tm_wday nor tm_yday unless
corresponding format options are used.  That means we must call timegm()
to set them.
2017-05-21 01:58:18 -05:00
Nicolas Williams 1900c7bcac Add private my_timegm() 2017-05-21 01:57:55 -05:00
Nicolas Williams 578d536233 Fix HAVE_TM_TM_GMT_OFF usage 2017-05-21 01:53:00 -05:00
Nicolas Williams b0d6d28310 Use AC_CHECK_MATH_FUNC() for all math functions 2017-05-21 01:22:39 -05:00
Nicolas Williams 10d66051ef Fix jv_load_file() assertion (fix #1410) 2017-05-20 18:53:48 -05:00
Nico Williams 6d89e297fe Add JQ_COLORS env var for color config (fix #1252) 2017-04-29 14:00:52 -04:00
Nicolas Williams 32d8f2000a Fix build 2017-04-26 20:55:50 -05:00
Nicolas Williams 0bd7614bd5 Always use jv_mem_*alloc() 2017-04-21 18:41:06 -05:00
Nicolas Williams 18753cb7a7 Also fix jn/2 and yn/2 2017-03-30 11:46:03 -05:00
Nicolas Williams 71d4ff6ed8 Fix frexp, modf, lgamma_r (fix #1374) 2017-03-29 00:07:42 -05:00
Nicolas Williams 7935d307f0 Allow . as {$a:$b} destructuring 2017-03-26 05:36:23 -05:00
Nicolas Williams 38bfec1594 Add flex/bison outputs 2017-03-26 05:36:23 -05:00
Nicolas Williams 2421dfb843 Add more destructuring tests 2017-03-26 05:36:22 -05:00
Nicolas Williams b142d484d5 Conditional exprs are not path exprs (fix #1368)
The conditional expression in if-then-elif-else-end cannot contribute to
path expressions because it doesn't change the input to any of the then/
elif/else expressions.  These must be generated via gen_subexp().

See also #1366.
2017-03-15 01:07:37 -05:00
Nicolas Williams 674e9fb7c2 Make and build dist in travis-CI (#1356) 2017-03-04 23:14:36 -06:00
Nicolas Williams 6878123202 Make configure.ac work for shallow clones 2017-03-04 22:57:35 -06:00
Nicolas Williams f7bf9a8e96 Make scripts/version work in shallow clones
Shallow clones don't have tags.  Use git describe --always --dirty in
such cases.

XXX Add branch name.
2017-03-03 14:30:58 -06:00
Nicolas Williams 89809a2d0f Fix make dist 2017-03-01 23:01:08 -06:00
Nicolas Williams 9df19f53e4 Fix off-by-one bug in #1108 fix 2017-03-01 22:45:02 -06:00
Nicolas Williams 65cbaac344 Array slice overflows (fix #1108) 2017-03-01 17:48:45 -06:00
Nicolas Williams 2fcb257168 getpath/1 should be a path expression (fix #1358)
It needs to be possible to do something like

    getpath($paths[]) += 1

meaning: increment all the paths in . that are listed in $paths[].

In order to do this getpath() needs to update the jq->path and
jq->value_at_path as necessary.
2017-02-28 21:54:30 -06:00
Nicolas Williams ea90b947b0 Fix warning about environ 2017-02-26 18:22:30 -06:00
Nicolas Williams 9a0d796298 Fix --without-oniguruma build 2017-02-26 18:22:23 -06:00
Nicolas Williams 0337027d54 Fix environ Win32 build problem 2017-02-26 18:06:35 -06:00
Nicolas Williams 8ea21a54ad Add halt, halt_error builtins (fix #386) 2017-02-26 16:34:56 -06:00
Nicolas Williams 6bac4ed059 Document stderr 2017-02-26 16:34:56 -06:00
Nicolas Williams a03ae02f44 Fix memory leak 2017-02-26 16:33:50 -06:00
Nicolas Williams 45f126514c Fix Appveyor for non-master branches 2017-02-26 16:13:22 -06:00
Nicolas Williams b279713e47 fixup 2017-02-26 16:12:57 -06:00
Nicolas Williams 3504bd9a2a Add $ENV builtin variable to access environment 2017-02-26 00:39:24 -06:00
Nicolas Williams 1a8a0ae0f5 Fix error message for undefined vars 2017-02-26 00:37:28 -06:00
Nicolas Williams 441b066dd8 Preserve precious vars in configure.ac (fix #795) 2017-02-25 21:49:38 -06:00
Nicolas Williams ef87010059 Document $ARGS and friends 2017-02-25 19:21:06 -06:00
Nicolas Williams 66fb962a66 Add $ARGS, --args and --jsonargs (fix #1345) 2017-02-25 19:13:28 -06:00
Nicolas Williams e8abc0a2f6 jq_compile_args(): allow object args to be object 2017-02-25 19:13:28 -06:00
Nicolas Williams 7b1bd99de6 Add jv_object_has() 2017-02-25 19:13:28 -06:00
Nicolas Williams 8619c6e060 Make test/shtest test of constant folding robust
Rather than testing that a constant expression produces so many
instructions, test that a variety of of such expressions produce the
same number of instructions.  This will make future changes in the
compiler less likely to break this test.
2017-02-25 19:13:28 -06:00
Nicolas Williams 2e1f2db8b5 Improve Appveyor build
Among other things, make TESTS=... breaks when using modules/oniguruma,
so set SUBDIRS= then.
2017-02-25 18:57:38 -06:00
Nicolas Williams ba32b33796 Windows: configure fails to detect missing math
For some reason AC_CHECK_MATH_FUNC() on Windows is not detecting a
handful of math functions as not being supported.  This commit is a bit
of a hack.
2017-02-25 18:51:09 -06:00
Nicolas Williams 6e77b017c5 Make comment consistent with code 2017-02-25 18:51:09 -06:00
Nicolas Williams 9801de6979 Windows: Also use -lshlwapi for jq(1) 2017-02-25 14:49:20 -06:00
Nicolas Williams c695f2cd9c When using builtin Oniguruma, set flags correctly
This should fix the Travis-CI build.
2017-02-25 14:11:42 -06:00
Nicolas Williams 27c29aeb7b Document localtime and strflocaltime 2017-02-23 23:39:24 -06:00
Nicolas Williams 06f20603f6 Add localtime and strflocaltime (fix #1349) 2017-02-23 22:26:53 -06:00
Nicolas Williams 607a9e3912 Improve manual section on assignment forms 2017-02-23 22:17:54 -06:00
Nicolas Williams d3b4ad04f5 Revert e7caf68 for Dockerfile 2017-02-23 22:01:43 -06:00
Nicolas Williams e7caf68edd Attempt to use builtin Oniguruma for CIs 2017-02-23 00:33:08 -06:00
Nicolas Williams c6374b6a1f Allow ./configure --with-oniguruma=builtin
This allows one to force the use of jq's builtin copy of Oniguruma
(builtin as a git submodule).

We still need to fix make dist to include that copy of Oniguruma.
2017-02-23 00:30:13 -06:00
Nicolas Williams 8cde328007 Allow var bindings in path expressions (#1347)
The expression in a variable binding is not and must not be seen as
being part of a path expression in which the variable binding appears.
2017-02-21 18:11:35 -06:00
Nicolas Williams 9b2179089b Improve manual 2017-02-15 11:48:39 -06:00
Nicolas Williams 597c1f6667 Add more missing math functions 2017-02-04 00:11:46 -06:00
Nicolas Williams 125071cf00 Fix handling of unsupported math functions 2017-02-04 00:11:10 -06:00
Nicolas Williams 2fb099e4cf Promote wiki more in README 2017-01-31 16:13:32 -06:00
Nicolas Williams 6f9646a44f Improve docs somewhat, inspired by #1326 2017-01-30 14:11:05 -06:00
Nicolas Williams bd7b48c1b9 Make |= delete LHS when RHS is empty (Fix #1314)
Now that #1313 is fixed, |= no longer outputs null when the RHS update
expression outputs empty.

When a user wants to keep the current value of the LHS they would have
the RHS update expression output `.`, so having `empty` achieve the same
thing would be redundant.  The obvious thing to do is to delete the LHS
when the RHS update outputs `empty` (i.e., doesn't output any values).

It's reasonable to think that existing programs won't be broken by this
change, because reduce and |= not handling empty well is clearly a bug.
(Though it's possible that some programs were using empty to quickly
terminate reduce or |=, it's not likely.  They should use label/break
instead.)

Prior to this change |= would use the _last_ value output by the RHS
update expression.  With this change |= will use the _first_ value
instead.  This change _is_ a minor backwards-incompatible change.  It
may or may not be acceptable; we'll see.  It is a useful change in that
it makes |= faster when the update expression produces multiple values.
2017-01-30 14:11:05 -06:00
Nicolas Williams aac8132f47 recurse (i.e., ..) needs to find leaf nulls 2017-01-30 14:11:05 -06:00
Nicolas Williams 7bdf3a39ad Add SQL-style operators (#1322) 2017-01-30 14:11:05 -06:00
Nicolas Williams 396543191c Make first(g) more efficient: extract only 1 value
first(g) was extracting two values, which, if g is slow, made first(g)
slow.  And if the second extraction were to throw an error, then
first(g) would throw that error, which is clearly not the right thing to
do.  Besides that, first(g) was allocating garbage, which it no longer
does.
2017-01-30 14:11:05 -06:00
Nicolas Williams e24af3c78e reduce: handle empty updates (fix #1313) 2017-01-30 14:11:05 -06:00
Nicolas Williams 0b8218515e Disable appveyor build for now 2016-03-21 18:59:52 -05:00
Nicolas Williams 1740fd036d Travis-CI build for OSX (fix #1083) 2016-01-19 00:19:24 -06:00
Nicolas Williams 6c635998f8 Always test sort_by/group_by post-#1082 2016-01-18 12:17:58 -06:00
Nicolas Williams d228490162 Add --enable-ubsan (undefined behavior sanitizer) 2016-01-17 13:06:27 -06:00
Nicolas Williams b92a12cf6c Run failing tests on Windows; ignore failures 2016-01-17 12:46:57 -06:00
Nicolas Williams c514368f98 Make it possible to run tests with set -x
One can now run:

    make TRACE_TESTS=1 check

to get detailed output
2016-01-17 12:45:01 -06:00
Nicolas Williams 062f86a625 Enable more tests on Windows; move sort_by tests
Use %%FAIL IGNORE MSG for error messages that differ on Windows, and
move the sort_by/group_by tests that fail qhen qsort() is not stable.

Now only tests/sorttest and tests/shtest are disabled on Windows.
2016-01-17 12:45:01 -06:00
Nicolas Williams c3d13260a7 Tweak appveyor.yml; disable failing tests 2016-01-17 01:49:50 -06:00
Nicolas Williams 093d20c3b9 Fix cross-compilation build 2016-01-16 16:51:06 -06:00
Nicolas Williams ec07936888 Fix Windows build (fix #911) 2016-01-16 16:51:01 -06:00
Nicolas Williams 8e4724e16c fixup non-maintainer build 2016-01-15 16:06:18 -06:00
Nicolas Williams 3afdbb7b7d Always build src/builtin.inc (fix #1069) 2016-01-15 11:59:14 -06:00
Nicolas Williams f217fa26ae MultiByteToWideChar() usage bug (fix #1072) 2016-01-15 11:58:41 -06:00
Nicolas Williams 8f6f28c8d3 Make it easier to use jq with shebangs (fix #1044)
Allow a continuation on a comment immediately after a shebang to make
this traditional hack possible:

    #!/bin/sh
    # this next line is ignored by jq \
    exec jq -f "$0" "$@"
    # jq code follows

But continue only on the first line following a shebang, and only if
it's a comment.
2015-12-14 22:08:17 -08:00
Nicolas Williams 856a4b2f3f Add note about IEEE754 to help msg 2015-12-15 00:00:03 -06:00
Nicolas Williams ec583deefa Fix JV_OBJECT() and go up to 18 2015-12-01 23:27:19 -06:00
Nicolas Williams 5fe053671a Assume . if either stdin/on isatty() (fix #1028)
And move the #define of isatty on Windows up.
2015-12-01 23:27:19 -06:00
Nicolas Williams 8f115e9eed Add address sanitizer (ASAN) support (fix #1002) 2015-10-27 20:06:21 -05:00
Nicolas Williams 8eb1367ca4 Heap buffer overflow in tokenadd() (fix #105)
This was an off-by one: the NUL terminator byte was not allocated on
resize.  This was triggered by JSON-encoded numbers longer than 256
bytes.
2015-10-24 17:27:56 -05:00
Nicolas Williams aa279ae7ac Fix builtin.o dependency (#992) 2015-10-22 22:33:47 -05:00
Nicolas Williams 4060535f9f Fix linux32 jq-1.5 executable (fix #935) 2015-10-22 22:07:34 -05:00
Nicolas Williams 1847333342 Rename sig/v1.5/jq-linux32.asc (prep for #935) 2015-10-22 22:06:27 -05:00
Nicolas Williams 87c278c8a5 Remove setup.sh (fix #993) 2015-10-22 21:53:53 -05:00
Nicolas Williams 49b8382c5f Remove expired key from KEYS 2015-10-12 19:11:52 -05:00
Nicolas Williams 583ab589dc Add KEYS file 2015-10-12 19:01:02 -05:00
Nicolas Williams c4524da3e0 EOF after newline in string mishandled (fix #951) 2015-09-22 20:11:54 -07:00
Nicolas Williams f57743eb37 Remove jq.1.default and related content 2015-08-23 23:08:11 -05:00
Nicolas Williams b6e3657eda Document getpath/setpath 2015-08-16 20:32:01 -05:00
Nicolas Williams a5b2c55a70 Fix streaming docs errors (fix #904) 2015-08-13 23:38:14 -05:00
Nicolas Williams b119853aef Test main.c fix for #817 2015-08-13 09:08:37 -07:00
Nicolas Williams 5d08d54636 Make search path expansion portable (fix #817)
This is part 2 of the fix to #817.  Part 1 is the previous commit.
2015-08-13 09:08:37 -07:00
Nicolas Williams 10df52efed Make all global module paths absolute (fix #817)
This is part 1 of the fix for #817.  The next commit is part 2.
2015-08-13 09:08:37 -07:00
Nicolas Williams c426d494f0 Improve assignment docs (see #897) 2015-08-11 18:05:22 -05:00
Nicolas Williams 9a3f802d20 Fix #896, double-free in setpath 2015-08-11 17:46:15 -05:00