Commit Graph
380 Commits
Author SHA1 Message Date
taoky 8f81668014 Fix the default colors to use 39, the default foreground color (#2904) 2023-09-22 09:18:41 +09:00
Nicolas Williams 11c528d04d Add setlocale() call (fix #1740) 2023-09-06 14:05:10 -05:00
Nicolas Williams 0e067ef936 Improve handling of non-integer numeric indices (fix #2815) 2023-08-27 12:32:12 +02:00
itchyny 7d64381016 Fix overflow on numeric comparison (#2849)
Although #2839 fixed the overflow of exponent subtraction,
there still is possibility of overflow in the `D2U` macro.
This patch fixes the overflow in the `D2U` macro, and also
truncates the maximum digits to `DEC_MAX_DIGITS`.
2023-08-17 15:56:57 +09:00
itchyny 0733fd3d58 Add a regression test for negative indices and fix a pick/1 test 2023-08-16 17:49:44 -05:00
itchyny 3fa10e8cc1 Fix crash on numeric comparison again (ref #2825)
The decNumber library subtracts the exponents of two numbers,
we make sure to limit the number of digits not to make it overflows.
Since the maximum adjusted exponent is `emax` and the minimum is
`emin - digits + 1`, we follow `emax - (emin - digits + 1) <= INT32_MAX`.
2023-08-13 14:17:25 +02:00
Alex Wilson 65ed95c93a Expose the names of a module's defined function via modulemeta
This reuses the existing `block_list_funcs` capability and adds an extra field on the `modulemeta` output, called `defs`, containing that list of functions.
2023-08-12 20:46:36 -05:00
Mattias WadmanandNicolas Williams fdab39bc7b Build windows 64bit binary using UCRT64 (#2832)
* Build windows 64bit binary using UCRT64

Is the default and recommended msystem setting. Will produce
binaries that are compatible with windows 10 and later.

Also run tests for 32bit build.

Related to #2831

* Use jq -b in tests/shtest

* Add Windows strptime

* Make Windows-optional tests not run on Windows again

---------

Co-authored-by: Nicolas Williams <nico@cryptonector.com>
2023-08-12 20:18:45 +02:00
itchyny a692060129 Change the default color of null to Bright Black 2023-08-05 23:21:12 -05:00
Emanuele Torre f94a9d463f Let error(null) throw null
This patch removes the weird behaviour of jv_invalid_with_msg(jv_null())
that returns jv_invalid() (i.e. empty), instead of a boxed jv_null().

The previous behaviour of  null|error  was obviously unintentional, and
allowing is jv_invalid_with_msg() to return values on which you can't
call jv_invalid_get_msg() is only error prone.
2023-08-04 17:38:52 -05:00
Nicolas Williams ddef804945 Clarify the // operator (close #2189) 2023-08-03 14:41:53 -05:00
itchyny 949d38e6dc Fix crash on numeric comparison (ref #2804) (#2818) 2023-08-03 23:19:52 +09:00
itchyny fd0d475694 Fix a test case added in #2790 2023-08-02 22:33:09 +09:00
itchynyandLeonid S. Usov 680baeffeb Fix rounding small (but not too small) numbers to zero on calculation
Co-authored-by: Leonid S. Usov <leonid.s.usov@gmail.com>
2023-08-02 00:32:32 -05:00
Nicolas Williams 27a4d5757e Require a main program (fix #2785) 2023-08-01 23:06:01 -05:00
pkoppstein 4d4c17ca6a Simple and efficient implementation of walk/1 (#2795) 2023-08-01 07:54:21 +09:00
itchyny 0f80921268 Fix constant folding of division and reminder with zero divisor
Previously constant folding of zero division (e.x. 1/0) produces a
compile error. This was incorrectly implemented by checking if the
division result is infinite, so produces wrong results compared to the
query where no constant folding is processed (e.x. 1e308/0.1). This
patch delays the operation when the divisor is zero. This makes the
results more consistent, but changes the exit code on zero division from
3 to 5. Also 0/0 now produces the zero division error, not NaN.

This patch also fixes the modulo operation. Previously constant folding
logic does not take care of the % operator, but now it folds if the both
dividend and divisor are safe numbers to cast to the integer type, and
the divisor is not zero. This patch also fixes some code that relies on
undefined cast behaviors in C. The modulo operation produces NaN if
either the dividend or divisor is NaN.
2023-07-31 15:39:13 -05:00
pkoppstein 336c3deb8a Test negative indices in path expressions 2023-07-28 12:34:26 -05:00
Emanuele Torre 17889a1a85 Fix memory leak for { $foo: bar }
{ BINDING: ExpD } wasn't freeing BINDING.

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60865
2023-07-27 20:59:35 -05:00
itchyny a1e791acf8 Rename --nul-output to --raw-output0, abort on string containing NUL
The option naming --nul-output was confusing, especially when we have a
similar option for input stream in the future (--nul-input vs --null-input).
Based on the observation of other command line tools, we rename the option
to --raw-output0. We also drop the short option -0 to avoid confusion on
introducing the NUL-delimited input option.

Unlike the other command line tools outputting file names with NUL delimiter,
jq deals with JSON, and its strings may contain NUL character. To protect
users from the risk of injection attacks, we abort the program and print an
error message before outputting strings including NUL character. Closes #2683.
2023-07-26 23:55:00 -05:00
pkoppstein 13fbe98dff manual.yml: remove contingent tests
Also clarify non-prescriptive nature of some tests in jq.test
2023-07-25 23:43:57 -05:00
pkoppstein 7b725378b9 builtin.jq: naive abs/0
manual.yml explains that the def is naive, and mentions fabs, etc.
2023-07-25 23:43:57 -05:00
Mattias Wadman a949745059 implode: Better invalid input validation and handling
Error on non-number and nan codepoint, would asserd before
Replace negative codepoint and surrogate range with unicode replacement character, would assert before

Fixes #1160
2023-07-25 13:38:41 -05:00
pkoppstein 8f49600a6b builtin.jq: simpler and faster transpose (#2758) 2023-07-25 09:12:23 +09:00
Mattias Wadman 16ae275d4c Fix leaks on invalid input for strftime, strflocaltime and mktime 2023-07-24 16:45:43 +02:00
David Korczynski 97c6d2848c Add fuzzer targeting load_file
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-24 15:45:12 +02:00
David Korczynski cf3c11bde8 Add compile fuzzer for OSS-Fuzz
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-24 13:44:27 +02:00
itchyny ed334b536f Improve manual in various ways (inputs, sort_by, foreach sections, etc.) (#2744)
- Add error/0 and mentions null input behavior (close #2231)
- Explain value iterator suffix syntax .foo[] (close #1047)
- Mention array slicing is also zero-based (close #2094)
- Add examples of input and inputs filters (close #2216, close #2470)
- Improve sort_by about multiple values (close #2103, close #2467, close #2474)
- Improve foreach section and simplify examples (close #1148, close #2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close #2036, close #2412)
- Add non-string examples of index/1, rindex/1 (close #1422)
- Simplify the example of truncate_stream/1 (close #1736)
2023-07-24 20:24:44 +09:00
Nico Williams 1cf6515c06 Fix try/catch catches more than it should #1859
Close #1885, #2140, #2011, #2220, #2485, #2073

Rename the FORK_OPT opcode to TRY_BEGIN, add a TRY_END opcode, and wrap
errors when raising through a TRY_END so that they will not be caught by
the matching TRY_BEGIN.

Now a `try exp catch handler` expression generates code like:

    TRY_BEGIN handler
    <exp>
    TRY_END
    JUMP past_handler
    handler: <handler>
    past_handler:
    ...

On backtrack through TRY_BEGIN it just backtracks.

If anything past the whole thing raises when <exp> produced a value,
then the TRY_END will catch the error, wrap it in another, and
backtrack.  The TRY_BEGIN will see a wrapped error and then it will
unwrap and re-raise the error.

If <exp> raises, then TRY_BEGIN will catch the error and jump to the
handler, but the TRY_BEGIN will not stack_save() in that case, so on
raise/backtrack the TRY_BEGIN will not execute again (nor will the
TRY_END).
2023-07-24 13:20:21 +02:00
itchyny 98f709d0c1 Fix stderr to output string with no decoration (fix #2063) 2023-07-23 17:06:00 -05:00
Nicolas Williams f64ef1a7d7 Fix --stream-errors w/o --stream, and doc+test (close #2748) 2023-07-23 03:33:28 -05:00
itchyny c9c45d7f03 Add tests for --arg, --argjson, and similar options 2023-07-22 12:29:24 -05:00
itchyny 99a77f724b Add a regression test for #2000 2023-07-21 09:01:28 +09:00
pkoppstein 3ec66c858c manual.yml: revise section on identity, clarifying what is subject to change
Mostly clarifications w.r.t. numbers.

The anomalous behavior of leading minus signs is documented, with an explicit notice that this will probably change.
2023-07-19 22:32:24 -05:00
itchyny 0615d4982c Respect NO_COLOR environment variable to disable color output 2023-07-19 00:05:41 -05:00
0b558f6ed4 Make object key color configurable (close #1739, #1791, #2638)
Co-authored-by: David Haguenauer <ml@kurokatta.org>
Co-authored-by: Eric Pruitt <eric.pruitt@gmail.com>
2023-07-18 17:53:29 -05:00
Emanuele Torre 15fd31de78 Define examples in the "examples" field instead of "example"
Only examples defined in the "examples" field are generated, if they are
defined in "example", they are just ignored.

Also fix a bunch of incorrect tests.
2023-07-18 12:12:50 -05:00
Emanuele Torre 01dfd8b86d Populate captures also for zero-width matches
Instead of just using {"captures":[]}.

sub functions are use captures for replacement expressions.
If we don't populate, captures for empty matches, the replacement
expression is run with an empty object as input instead of an object
containing the named captures with "" as value:

* before:

  $ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
  "ZfooZbarZnull"

* after:

  $ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
  "ZfooZbarZ"

---

I also removed a redundant

  result = NULL;
  if (result) {
    ...
  }
2023-07-18 12:12:50 -05:00
Emanuele Torre bb0f898c01 Initialise jq_state->debug_cb{,_data} to NULL
To make debug/0 not call an uninitialised function pointer when using
--run-tests or when using a jq_state on which jq_set_debug_cb() has not
been called.
2023-07-18 12:12:50 -05:00
itchyny 9e4c71558a Split man.test to make tests pass without oniguruma (#2722) 2023-07-18 08:17:21 +09:00
Emanuele Torre 4b3090a9ab Make stream parser error when a non-empty object or array is used as key
Fixes #2463
2023-07-17 13:15:55 -05:00
Emanuele Torre e79335e3a5 Initialise jq_state->input_cb{,_data} to NULL in jq_init()
To avoid causing segmentation faults when  input/1  is called in a
jq_state on which  jq_set_input_cb()  has not been called; e.g. the one
used by  jq --run-tests.

That segfault could also be fixed in run_jq_tests() by calling:

  jq_set_input_cb(jq, NULL, NULL);

But I think it makes sense to just make jq_init() initialise those
values to NULL.

Ref: https://github.com/jqlang/jq/pull/2717#discussion_r1264338841
2023-07-16 00:03:46 -05:00
Emanuele Torre 1915b689f7 Fix indices/1 and rindex/1 in case of overlapping matches in strings
Fixes #2433
2023-07-14 23:40:37 -05:00
Emanuele Torre 4b1ac7c95f Parse nan in JSON as NaN instead of triggering a parse error (#2712)
* Fix memory leak for has(nan)

jv_array_get() used to be responsible of freeing the input array, but
since b5c4c3d67d, it is no longer called
if the key is nan. We need to free it manually to avoid leaking the
array.

* Parse  nan  in JSON as NaN instead of triggering a parse error

Fixes #2021
2023-07-14 07:12:27 +09:00
itchyny 092fef740a Make sure distcheck passes and run dist job on Pull Request 2023-07-13 10:20:04 -05:00
itchyny 689a4e660b Fix uri format to follow RFC 3986 (#2157) 2023-07-13 23:23:23 +09:00
itchyny b5c4c3d67d Fix has(nan) on arrays to output false 2023-07-12 09:40:05 -05:00
Emanuele Torre c99981c5b2 Update "tests/man.test" and remove superflous whitespace in manual.yml
tests/man.test was generated incorrectly and had some duplicate tests,
the new CI workflow didn't like that.
2023-07-11 00:24:50 -05:00
Nicolas Williams 1ec8be1422 Update tests/man.test 2023-07-10 14:18:27 -05:00
davkor 3df8f90c4e Add first fuzzer for integration with OSS-Fuzz.
Signed-off-by: David Korczynski <david@adalogics.com>
2023-07-10 13:57:38 -05:00