When the UPDATE query in a reduce or foreach syntax does not emit
a value, it should simply skip updating the state variable. But
currently it resets the state variable to null.
This comment predates the addition of $ARGS in jq 1.6, so there was no
way to access named arguments with an invalid identifier. Rather than
forbid such names, update the manual to clarify.
When jq is not supplied with a program, it usually exits with an error
and usage text, except for when jq is not connected to a TTY (i.e., not
interactive), in which case a default program of `.` is used. Combined
with `-f`, it then tries to read `.` as the filename.
$ jq
jq - commandline JSON processor [version 1.7.1]
…
$ jq -f
jq - commandline JSON processor [version 1.7.1]
…
$ jq | cat
^C
$ jq -f | cat
jq: Could not open .: It's a directory
Fixes:
* Short options cannot be repeated in the same group:
e.g., `-nn` fails, but `--null-input --null-input` works
* Short options are processed in an arbitrary fixed order, not the order
given by the user: e.g., `-hV`, `-Vh`, `-h -V`, and `--help --version`
are processed as `-h`, but `-V -h` and `--version --help` are
processed as `-V`
* `-L` cannot be a part of a short option group (as the last one):
e.g., `-nLlib` and `-nL lib` fail
I also changed the behavior of limit/2 with negative count to emit an error, like nth/2 does.
Also, I redefined nth/2 using skip/2 to minimize the impact of growth of builtin filters.
This check was redundant and useless since jv2tm() used to require 8
elements, and errored anyway if not enough elements were provided.
After commit bc96146, this check also prevents mktime/0 from accepting
now supported inputs with less than 8 elements.
Ref: https://github.com/jqlang/jq/pull/3070#issuecomment-2299902072
Reported-by: itchyny <itchyny@cybozu.co.jp>
`jq_reset` calls `jv_free` on the `exit_code` and the `error_message` stored on the jq state.
However, it doesn't replace the actual instance of those members. This means that subsequent
calls to `jq_reset` will call `jv_free` again on those members, which in turn may call `free`
on the same pointer multiple times. Freeing the same pointer multiple times is undefined
behavior and can cause heap corruption, which is how I spotted this issue.
In practice, this issue only occurs when using a program that may `halt_error`, because that
is when the `exit_code` and `error_message` are set to values other than `jv_invalid`.
Subsequent attempts to call `jq_start` (which calls `jq_reset` internally) after hitting a
`halt_error` can cause you to run into this issue.
The changes simply reset the `exit_code` and the `error_message` to `jv_invalid` (the initial
value set in `jq_init`) after they are freed.
`-b`/`--binary` should be accepted on all platforms so that they can be used in
code that is meant to be portable.
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
arrays[arrays] is a special case of "INDEX" that actually returns an
array containing the indices in which the array that is being indexed
contains the start of the key array.
So array keys, for array values, are a kind of key that can be "got",
but not "set". jv_setpath() was not freeing the value it "got" from
indexing that key, in case the following "set" on that key failed,
resulting in a leak.
$ ./jq -n '[] | setpath([[1]]; 1)'
jq: error (at <unknown>): Cannot update field at array index of array
=================================================================
==953483==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 272 byte(s) in 1 object(s) allocated from:
#0 0x725f4d4e1359 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x5ec17b1a7438 in jv_mem_alloc src/jv_alloc.c:141
SUMMARY: AddressSanitizer: 272 byte(s) leaked in 1 allocation(s).
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66061
Now, time functions accept array inputs even if they don't have all the
elements, 0 will be assumed if a value is not present.
Also, jv2tm now properly clamps large number values to a signed 32-bit
integer and rejects nan.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65885
tests/mantest were failing on Raspberry Pi OS 11:
==10205== Conditional jump or move depends on uninitialised value(s)
==10205== at 0x48806F8: checkfail (jq_test.c:58)
==10205== by 0x48806F8: run_jq_tests (jq_test.c:99)
==10205== by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205== by 0x10B37F: main (main.c:576)
==10205==
==10205== Conditional jump or move depends on uninitialised value(s)
==10205== at 0x4880714: checkfail (jq_test.c:58)
==10205== by 0x4880714: run_jq_tests (jq_test.c:99)
==10205== by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205== by 0x10B37F: main (main.c:576)
In windows, time_t is a signed 32-bit integer type, so TIME_MAX needs to
be declared as INT32_MAX instead of INT64_MAX.
Also bump NetBSD's strptime to revision 1.65 from 1.63 to fix undefined
behaviour (signed integer overflow) bugs.
Related NetBSD problem report:
https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58041
Noticed thanks to a compiler warning in the windows build CI.
I declared the d variable as time_t instead of as unsigned to ensure
that the signedness/size of TIME_MAX-d in the sse > TIME_MAX-d check
is always correct, and to prevent -Wsign-compare warnings from
triggering in the windows build.
There was a incorrect else, that caused jq to not ensure that the
argument to strftime/1 is a string when the input is a number; this ends
up calling jv_string_value on a non-string value, which does not work,
and causes an assert failure.
Also fix same bug in strflocaltime/1.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67403
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
Before, arrays would not reset colors after `[` and `,`, but objects
would; it would reset colors twice before `]` and `}`; and some cases of
indentation would have colors applied. Now, colors are reset immediately
after any token that is colored, before any indentation. This makes the
formatting consistent, for the benefit of custom `JQ_COLORS`.
C99 only allows string literals long at most 4095 characters.
jq_builtins was a lot longer than that.
I rewrote all the optional libm error stubs in C so the value of
jq_builtins is not build dependent.
I replaced the command that generates builtin.inc with a POSIX compliant
od|sed command that encodes builtin.jq as a comma delimited list of
octal numbers (that can be embedded in C using a {} literal).
I also added -Woverlength-strings to AM_CFLAGS to verify that the
problem is fixed.
Fixes#1481
Previously, ltrimstr/rtrimstr would just let the input pass through for
non-string inputs or arguments.
That was happening because, they were leaking the errors returned by
startswith/endswith treating them as if they were jv_false().
The leak was resolved by #2977 for 1.7.1
This patch rewrites ltrimstr and rtrimstr in jq, and makes them not
ignore startswith and endswith errors anymore.
decNumberToString calls for a buffer that can hold a string of digits+14
characters, not a buffer of size digits+14.
We need to allocate an extra byte for the NUL byte.
-10E-1000010001, for example, will be stringified as -1.0E-1000010000
and decNumberToString will currently write an extra NUL byte after the
allocated buffer in the heap.
Originally reported by @SEU-SSL on GitHub.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64574
Fixes GHSA-686w-5m7m-54vc
The unit allocated for decNumberCompare was accidentally removed by
commit 680baeffeb (PR #2804)
This caused a stack overflow when comparing a nan with a payload of 1000
or more.
This bug was found by OSS-fuzz.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64771
Fixes GHSA-7hmr-442f-qc8j
It also fixes 1e999999999 > 1e-1147483646 triggering UBSAN errors
Fixes#2968
itchyny