The option naming --nul-output was confusing, especially when we have a
similar option for input stream in the future (--nul-input vs --null-input).
Based on the observation of other command line tools, we rename the option
to --raw-output0. We also drop the short option -0 to avoid confusion on
introducing the NUL-delimited input option.
Unlike the other command line tools outputting file names with NUL delimiter,
jq deals with JSON, and its strings may contain NUL character. To protect
users from the risk of injection attacks, we abort the program and print an
error message before outputting strings including NUL character. Closes#2683.
RS should only only have special meaning when parsing json-seq.
Before this fix, RS was sometimes treated as a value terminator, causing
weird results.
Bug discovered by OSS fuzz.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60835
It seems that bison doesn't call destructors for mid-rule action
components on error, since it does not know their type.
A mid-rule action was used to allocate the "text" string used as format
by string literals without a format, which would leak on error.
This patch replaces it with a new NoFormat component of type <literal>.
Now bison will call jv_free() on that string after a syntax error.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60836
$ ./jq '"'
jq: error: syntax error, unexpected end of file, expecting QQSTRING_TEXT or QQSTRING_INTERP_START or QQSTRING_END (Unix shell quoting issues?) at <top-level>, line 1:
"
jq: 1 compile error
=================================================================
==1495450==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 21 byte(s) in 1 object(s) allocated from:
#0 0x7fc21aee1359 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x557ccb6ff608 in jv_mem_alloc src/jv_alloc.c:141
SUMMARY: AddressSanitizer: 21 byte(s) leaked in 1 allocation(s).
Error on non-number and nan codepoint, would asserd before
Replace negative codepoint and surrogate range with unicode replacement character, would assert before
Fixes#1160
In previous code nomem_handler in jv_alloc.c is called only once
and therefore the structure is not initialized for second and
following threads.
This leads to segmentation fault in multi-threading environment.
This patch moves initialization of nomem_handler out of the
pthread_once call.
This makes the standard build instructions a bit easier,
just ./configure, and also requires less tools installed (bison).
Also i think few people probably want to generate the lexer and paser code.
valgrind (3.19.0) locally on macOS docker container seems to detect this leak but not CI.
$ cat modulemeta.test
modulemeta | .deps |= length
"c"
{"whatever":null,"deps":6}
$ valgrind --error-exitcode=1 --leak-check=full -q ./jq --run-tests < modulemeta.test
Test #1: 'modulemeta | .deps |= length' at line number 1
*** Insufficient results for test at line number 3: modulemeta | .deps |= length
0 of 1 tests passed (0 malformed, 0 skipped)
==14129== 482 (392 direct, 90 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 8
==14129== at 0x48407B4: malloc (vg_replace_malloc.c:381)
==14129== by 0x13BC78: jv_mem_alloc (jv_alloc.c:122)
==14129== by 0x1371C6: jvp_object_new (jv.c:1521)
==14129== by 0x145840: parse_token (jv_parse.c:167)
==14129== by 0x145840: scan (jv_parse.c:684)
==14129== by 0x145840: jv_parser_next (jv_parse.c:793)
==14129== by 0x1463CB: jv_parse_sized (jv_parse.c:865)
==14129== by 0x136BD9: run_jq_tests (jq_test.c:183)
==14129== by 0x1370FA: jq_testsuite (jq_test.c:34)
==14129== by 0x13135C: main (main.c:582)
==14129==
This is unnecessary since we are using AC_USE_SYSTEM_EXTENSIONS in
configure.ac, which makes the compiler always run with -D_GNU_SOURCE=1
when building for GNU/Linux.
Also, the #define _GNU_SOURCE in src/util.c, and in
src/inject_errors.c (--enable-inject-errors), were not guarded by a
#ifndef _GNU_SOURCE so they were just triggering redefinition warnings.
- Add error/0 and mentions null input behavior (close#2231)
- Explain value iterator suffix syntax .foo[] (close#1047)
- Mention array slicing is also zero-based (close#2094)
- Add examples of input and inputs filters (close#2216, close#2470)
- Improve sort_by about multiple values (close#2103, close#2467, close#2474)
- Improve foreach section and simplify examples (close#1148, close#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close#2036, close#2412)
- Add non-string examples of index/1, rindex/1 (close#1422)
- Simplify the example of truncate_stream/1 (close#1736)
Close#1885, #2140, #2011, #2220, #2485, #2073
Rename the FORK_OPT opcode to TRY_BEGIN, add a TRY_END opcode, and wrap
errors when raising through a TRY_END so that they will not be caught by
the matching TRY_BEGIN.
Now a `try exp catch handler` expression generates code like:
TRY_BEGIN handler
<exp>
TRY_END
JUMP past_handler
handler: <handler>
past_handler:
...
On backtrack through TRY_BEGIN it just backtracks.
If anything past the whole thing raises when <exp> produced a value,
then the TRY_END will catch the error, wrap it in another, and
backtrack. The TRY_BEGIN will see a wrapped error and then it will
unwrap and re-raise the error.
If <exp> raises, then TRY_BEGIN will catch the error and jump to the
handler, but the TRY_BEGIN will not stack_save() in that case, so on
raise/backtrack the TRY_BEGIN will not execute again (nor will the
TRY_END).
Nicolas Williams