Strip UTF-8 BOM at the start of each input file (#3657)

The UTF-8 BOM was stripped only at the start of the parser stream, but jq
feeds all input files into a single parser, so a BOM at the start of the
second and later files reached the parser as content and failed with a
confusing "Invalid numeric literal" error. Strip it in the file readers
instead, at the start of each input file including stdin, and in
jv_load_file.

This also strips the BOM for raw input, so -R and --rawfile no longer
yield a leading U+FEFF, and a BOM at the start of a -f program file or an
imported module is now accepted rather than reported as a syntax error.
In exchange, "\uFEFF1" | fromjson is now a parse error, since a U+FEFF
inside a string is content rather than an encoding marker, and the
dedicated "Malformed BOM" error is gone.

tests/jq.test

@@ -42,13 +42,6 @@ null
 {"x":-1}
 {"x":1}
 
-# The input line starts with a 0xFEFF (byte order mark) codepoint
-# No, there is no reason to have a byte order mark in UTF8 text.
-# But apparently people do, so jq shouldn't break on it.
-.
-"byte order mark"
-"byte order mark"
-
 # We test escapes by matching them against Unicode codepoints
 # FIXME: more tests needed for weird unicode stuff (e.g. utf16 pairs)
 "Aa\r\n\t\b\f\u03bc"

tests/shtest

@@ -57,6 +57,17 @@ for exp in '1+1' '1-1' '2*3' '9/3' '9%3' '9==3' '9!=3' \
   fi
 done
 
+## Test UTF-8 BOM is stripped at the start of each input file
+printf '\357\273\277[{"foo": 1}\n,{"foo": 2}\n]\n' > $d/bom1.json
+printf '\357\273\277[{"foo": 3}\n,{"foo": 4}\n]\n' > $d/bom2.json
+$VALGRIND $Q $JQ -e '.[0].foo == 1' $d/bom1.json
+$VALGRIND $Q $JQ -se '. == [[{"foo":1},{"foo":2}],[{"foo":3},{"foo":4}]]' $d/bom1.json $d/bom2.json
+$VALGRIND $Q $JQ -ne '[inputs] == [[{"foo":1},{"foo":2}],[{"foo":3},{"foo":4}]]' $d/bom1.json $d/bom2.json
+$VALGRIND $Q $JQ -ne '[inputs] == [[{"foo":1},{"foo":2}],[{"foo":3},{"foo":4}]]' $d/bom1.json - < $d/bom2.json
+$VALGRIND $Q $JQ -Rne '[inputs] == ["[{\"foo\": 1}", ",{\"foo\": 2}", "]", "[{\"foo\": 3}", ",{\"foo\": 4}", "]"]' $d/bom1.json $d/bom2.json
+$VALGRIND $Q $JQ -ne --slurpfile x $d/bom1.json '$x == [[{"foo":1},{"foo":2}]]'
+$VALGRIND $Q $JQ -ne --rawfile x $d/bom1.json '$x == "[{\"foo\": 1}\n,{\"foo\": 2}\n]\n"'
+
 ## Test JSON sequence support
 cat > $d/expected <<EOF
 jq: ignoring parse error: Truncated value at line 2, column 5