Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Update the built-in SQLite to the latest 3.45.0 beta. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
3d8d5c0fda083587c1e999021291ac25 |
User & Date: | drh 2024-01-12 20:40:06.305 |
Context
2024-01-30
| ||
18:04 | Update the built-in SQLite to version 3.45.1 (check-in: 796aa7e55a user: drh tags: trunk) | |
2024-01-12
| ||
20:40 | Update the built-in SQLite to the latest 3.45.0 beta. (check-in: 3d8d5c0fda user: drh tags: trunk) | |
2023-10-24
| ||
13:35 | Update the built-in SQLite to the first 3.44.0 beta. (check-in: 830ad9a734 user: drh tags: trunk) | |
Changes
Changes to tclsqlite3.c.
1 2 3 | #ifndef USE_SYSTEM_SQLITE /****************************************************************************** ** This file is an amalgamation of many separate C source files from SQLite | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | #ifndef USE_SYSTEM_SQLITE /****************************************************************************** ** This file is an amalgamation of many separate C source files from SQLite ** version 3.45.0. By combining all the individual C code files into this ** single large file, the entire code can be compiled as a single translation ** unit. This allows many compilers to do optimizations that would not be ** possible if the files were compiled separately. Performance improvements ** of 5% or more are commonly seen when SQLite is compiled as a single ** translation unit. ** ** This file is all you need to compile SQLite. To use SQLite in other ** programs, you need this file and the "sqlite3.h" header file that defines ** the programming interface to the SQLite library. (If you do not have ** the "sqlite3.h" header file at hand, you will find a copy embedded within ** the text of this file. Search for "Begin file sqlite3.h" to find the start ** of the embedded sqlite3.h header file.) Additional code files may be needed ** if you want a wrapper to interface SQLite with your choice of programming ** language. The code for the "sqlite3" command-line shell is also in a ** separate file. This file contains only code for the core SQLite library. ** ** The content in this amalgamation comes from Fossil check-in ** f47a5f4e0ce078e6cc1183e6cbb3c4013af3. */ #define SQLITE_CORE 1 #define SQLITE_AMALGAMATION 1 #ifndef SQLITE_PRIVATE # define SQLITE_PRIVATE static #endif /************** Begin file sqliteInt.h ***************************************/ |
︙ | ︙ | |||
456 457 458 459 460 461 462 | ** been edited in any way since it was last checked in, then the last ** four hexadecimal digits of the hash may be modified. ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ | | | | | 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 | ** been edited in any way since it was last checked in, then the last ** four hexadecimal digits of the hash may be modified. ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.45.0" #define SQLITE_VERSION_NUMBER 3045000 #define SQLITE_SOURCE_ID "2024-01-12 11:44:49 f47a5f4e0ce078e6cc1183e6cbb3c4013af379b496efae94863a42e5c39928ed" /* ** CAPI3REF: Run-Time Library Version Numbers ** KEYWORDS: sqlite3_version sqlite3_sourceid ** ** These interfaces provide the same information as the [SQLITE_VERSION], ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros |
︙ | ︙ | |||
2437 2438 2439 2440 2441 2442 2443 | ** to an ORDER BY clause, all fields required by the caller are present in the ** sorted records. However, if SQLite determines based on the declared type ** of a table column that its values are likely to be very large - larger ** than the configured sorter-reference size threshold - then a reference ** is stored in each sorted record and the required column values loaded ** from the database as records are returned in sorted order. The default ** value for this option is to never use this optimization. Specifying a | | | 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 | ** to an ORDER BY clause, all fields required by the caller are present in the ** sorted records. However, if SQLite determines based on the declared type ** of a table column that its values are likely to be very large - larger ** than the configured sorter-reference size threshold - then a reference ** is stored in each sorted record and the required column values loaded ** from the database as records are returned in sorted order. The default ** value for this option is to never use this optimization. Specifying a ** negative value for this option restores the default behavior. ** This option is only available if SQLite is compiled with the ** [SQLITE_ENABLE_SORTER_REFERENCES] compile-time option. ** ** [[SQLITE_CONFIG_MEMDB_MAXSIZE]] ** <dt>SQLITE_CONFIG_MEMDB_MAXSIZE ** <dd>The SQLITE_CONFIG_MEMDB_MAXSIZE option accepts a single parameter ** [sqlite3_int64] parameter which is the default maximum size for an in-memory |
︙ | ︙ | |||
2612 2613 2614 2615 2616 2617 2618 | ** ** [[SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE]] ** <dt>SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE</dt> ** <dd> Usually, when a database in wal mode is closed or detached from a ** database handle, SQLite checks if this will mean that there are now no ** connections at all to the database. If so, it performs a checkpoint ** operation before closing the connection. This option may be used to | | | 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 | ** ** [[SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE]] ** <dt>SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE</dt> ** <dd> Usually, when a database in wal mode is closed or detached from a ** database handle, SQLite checks if this will mean that there are now no ** connections at all to the database. If so, it performs a checkpoint ** operation before closing the connection. This option may be used to ** override this behavior. The first parameter passed to this operation ** is an integer - positive to disable checkpoints-on-close, or zero (the ** default) to enable them, and negative to leave the setting unchanged. ** The second parameter is a pointer to an integer ** into which is written 0 or 1 to indicate whether checkpoints-on-close ** have been disabled - 0 if they are not disabled, 1 if they are. ** </dd> ** |
︙ | ︙ | |||
4264 4265 4266 4267 4268 4269 4270 | ** <li> sqlite3_extended_errcode() ** <li> sqlite3_errmsg() ** <li> sqlite3_errmsg16() ** <li> sqlite3_error_offset() ** </ul> ** ** ^The sqlite3_errmsg() and sqlite3_errmsg16() return English-language | | > > | | > | 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 4276 4277 4278 4279 4280 4281 4282 4283 4284 4285 4286 4287 4288 | ** <li> sqlite3_extended_errcode() ** <li> sqlite3_errmsg() ** <li> sqlite3_errmsg16() ** <li> sqlite3_error_offset() ** </ul> ** ** ^The sqlite3_errmsg() and sqlite3_errmsg16() return English-language ** text that describes the error, as either UTF-8 or UTF-16 respectively, ** or NULL if no error message is available. ** (See how SQLite handles [invalid UTF] for exceptions to this rule.) ** ^(Memory to hold the error message string is managed internally. ** The application does not need to worry about freeing the result. ** However, the error string might be overwritten or deallocated by ** subsequent calls to other SQLite interface functions.)^ ** ** ^The sqlite3_errstr(E) interface returns the English-language text ** that describes the [result code] E, as UTF-8, or NULL if E is not an ** result code for which a text error message is available. ** ^(Memory to hold the error message string is managed internally ** and must not be freed by the application)^. ** ** ^If the most recent error references a specific token in the input ** SQL, the sqlite3_error_offset() interface returns the byte offset ** of the start of that token. ^The byte offset returned by ** sqlite3_error_offset() assumes that the input SQL is UTF8. |
︙ | ︙ | |||
5882 5883 5884 5885 5886 5887 5888 | ** are innocuous. Developers are advised to avoid using the ** SQLITE_INNOCUOUS flag for application-defined functions unless the ** function has been carefully audited and found to be free of potentially ** security-adverse side-effects and information-leaks. ** </dd> ** ** [[SQLITE_SUBTYPE]] <dt>SQLITE_SUBTYPE</dt><dd> | | | | | | > | > > > > > > > > > > > > > > | 5885 5886 5887 5888 5889 5890 5891 5892 5893 5894 5895 5896 5897 5898 5899 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910 5911 5912 5913 5914 5915 5916 5917 5918 5919 5920 5921 5922 5923 5924 5925 5926 5927 | ** are innocuous. Developers are advised to avoid using the ** SQLITE_INNOCUOUS flag for application-defined functions unless the ** function has been carefully audited and found to be free of potentially ** security-adverse side-effects and information-leaks. ** </dd> ** ** [[SQLITE_SUBTYPE]] <dt>SQLITE_SUBTYPE</dt><dd> ** The SQLITE_SUBTYPE flag indicates to SQLite that a function might call ** [sqlite3_value_subtype()] to inspect the sub-types of its arguments. ** This flag instructs SQLite to omit some corner-case optimizations that ** might disrupt the operation of the [sqlite3_value_subtype()] function, ** causing it to return zero rather than the correct subtype(). ** SQL functions that invokes [sqlite3_value_subtype()] should have this ** property. If the SQLITE_SUBTYPE property is omitted, then the return ** value from [sqlite3_value_subtype()] might sometimes be zero even though ** a non-zero subtype was specified by the function argument expression. ** ** [[SQLITE_RESULT_SUBTYPE]] <dt>SQLITE_RESULT_SUBTYPE</dt><dd> ** The SQLITE_RESULT_SUBTYPE flag indicates to SQLite that a function might call ** [sqlite3_result_subtype()] to cause a sub-type to be associated with its ** result. ** Every function that invokes [sqlite3_result_subtype()] should have this ** property. If it does not, then the call to [sqlite3_result_subtype()] ** might become a no-op if the function is used as term in an ** [expression index]. On the other hand, SQL functions that never invoke ** [sqlite3_result_subtype()] should avoid setting this property, as the ** purpose of this property is to disable certain optimizations that are ** incompatible with subtypes. ** </dd> ** </dl> */ #define SQLITE_DETERMINISTIC 0x000000800 #define SQLITE_DIRECTONLY 0x000080000 #define SQLITE_SUBTYPE 0x000100000 #define SQLITE_INNOCUOUS 0x000200000 #define SQLITE_RESULT_SUBTYPE 0x001000000 /* ** CAPI3REF: Deprecated Functions ** DEPRECATED ** ** These functions are [deprecated]. In order to maintain ** backwards compatibility with older code, these functions continue |
︙ | ︙ | |||
6092 6093 6094 6095 6096 6097 6098 6099 6100 6101 6102 6103 6104 6105 | ** METHOD: sqlite3_value ** ** The sqlite3_value_subtype(V) function returns the subtype for ** an [application-defined SQL function] argument V. The subtype ** information can be used to pass a limited amount of context from ** one SQL function to another. Use the [sqlite3_result_subtype()] ** routine to set the subtype for the return value of an SQL function. */ SQLITE_API unsigned int sqlite3_value_subtype(sqlite3_value*); /* ** CAPI3REF: Copy And Free SQL Values ** METHOD: sqlite3_value ** | > > > > > > | 6110 6111 6112 6113 6114 6115 6116 6117 6118 6119 6120 6121 6122 6123 6124 6125 6126 6127 6128 6129 | ** METHOD: sqlite3_value ** ** The sqlite3_value_subtype(V) function returns the subtype for ** an [application-defined SQL function] argument V. The subtype ** information can be used to pass a limited amount of context from ** one SQL function to another. Use the [sqlite3_result_subtype()] ** routine to set the subtype for the return value of an SQL function. ** ** Every [application-defined SQL function] that invoke this interface ** should include the [SQLITE_SUBTYPE] property in the text ** encoding argument when the function is [sqlite3_create_function|registered]. ** If the [SQLITE_SUBTYPE] property is omitted, then sqlite3_value_subtype() ** might return zero instead of the upstream subtype in some corner cases. */ SQLITE_API unsigned int sqlite3_value_subtype(sqlite3_value*); /* ** CAPI3REF: Copy And Free SQL Values ** METHOD: sqlite3_value ** |
︙ | ︙ | |||
6222 6223 6224 6225 6226 6227 6228 | ** SQLite is free to discard the auxiliary data at any time, including: <ul> ** <li> ^(when the corresponding function parameter changes)^, or ** <li> ^(when [sqlite3_reset()] or [sqlite3_finalize()] is called for the ** SQL statement)^, or ** <li> ^(when sqlite3_set_auxdata() is invoked again on the same ** parameter)^, or ** <li> ^(during the original sqlite3_set_auxdata() call when a memory | | > > > | | > > > > > | 6246 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6258 6259 6260 6261 6262 6263 6264 6265 6266 6267 6268 6269 6270 6271 6272 6273 6274 6275 | ** SQLite is free to discard the auxiliary data at any time, including: <ul> ** <li> ^(when the corresponding function parameter changes)^, or ** <li> ^(when [sqlite3_reset()] or [sqlite3_finalize()] is called for the ** SQL statement)^, or ** <li> ^(when sqlite3_set_auxdata() is invoked again on the same ** parameter)^, or ** <li> ^(during the original sqlite3_set_auxdata() call when a memory ** allocation error occurs.)^ ** <li> ^(during the original sqlite3_set_auxdata() call if the function ** is evaluated during query planning instead of during query execution, ** as sometimes happens with [SQLITE_ENABLE_STAT4].)^ </ul> ** ** Note the last two bullets in particular. The destructor X in ** sqlite3_set_auxdata(C,N,P,X) might be called immediately, before the ** sqlite3_set_auxdata() interface even returns. Hence sqlite3_set_auxdata() ** should be called near the end of the function implementation and the ** function implementation should not make any use of P after ** sqlite3_set_auxdata() has been called. Furthermore, a call to ** sqlite3_get_auxdata() that occurs immediately after a corresponding call ** to sqlite3_set_auxdata() might still return NULL if an out-of-memory ** condition occurred during the sqlite3_set_auxdata() call or if the ** function is being evaluated during query planning rather than during ** query execution. ** ** ^(In practice, auxiliary data is preserved between function calls for ** function parameters that are compile-time constants, including literal ** values and [parameters] and expressions composed from the same.)^ ** ** The value of the N parameter to these interfaces should be non-negative. ** Future enhancements may make use of negative N values to define new |
︙ | ︙ | |||
6270 6271 6272 6273 6274 6275 6276 | ** <li> An out-of-memory error occurs during the call to ** sqlite3_set_clientdata() which attempts to register pointer P. ** <li> A subsequent call to sqlite3_set_clientdata(D,N,P,X) is made ** with the same D and N parameters. ** <li> The database connection closes. SQLite does not make any guarantees ** about the order in which destructors are called, only that all ** destructors will be called exactly once at some point during the | | | 6302 6303 6304 6305 6306 6307 6308 6309 6310 6311 6312 6313 6314 6315 6316 | ** <li> An out-of-memory error occurs during the call to ** sqlite3_set_clientdata() which attempts to register pointer P. ** <li> A subsequent call to sqlite3_set_clientdata(D,N,P,X) is made ** with the same D and N parameters. ** <li> The database connection closes. SQLite does not make any guarantees ** about the order in which destructors are called, only that all ** destructors will be called exactly once at some point during the ** database connection closing process. ** </ul> ** ** SQLite does not do anything with client data other than invoke ** destructors on the client data at the appropriate time. The intended ** use for client data is to provide a mechanism for wrapper libraries ** to store additional information about an SQLite database connection. ** |
︙ | ︙ | |||
6503 6504 6505 6506 6507 6508 6509 6510 6511 6512 6513 6514 6515 6516 | ** The sqlite3_result_subtype(C,T) function causes the subtype of ** the result from the [application-defined SQL function] with ** [sqlite3_context] C to be the value T. Only the lower 8 bits ** of the subtype T are preserved in current versions of SQLite; ** higher order bits are discarded. ** The number of subtype bytes preserved by SQLite might increase ** in future releases of SQLite. */ SQLITE_API void sqlite3_result_subtype(sqlite3_context*,unsigned int); /* ** CAPI3REF: Define New Collating Sequences ** METHOD: sqlite3 ** | > > > > > > > > > > > > > > | 6535 6536 6537 6538 6539 6540 6541 6542 6543 6544 6545 6546 6547 6548 6549 6550 6551 6552 6553 6554 6555 6556 6557 6558 6559 6560 6561 6562 | ** The sqlite3_result_subtype(C,T) function causes the subtype of ** the result from the [application-defined SQL function] with ** [sqlite3_context] C to be the value T. Only the lower 8 bits ** of the subtype T are preserved in current versions of SQLite; ** higher order bits are discarded. ** The number of subtype bytes preserved by SQLite might increase ** in future releases of SQLite. ** ** Every [application-defined SQL function] that invokes this interface ** should include the [SQLITE_RESULT_SUBTYPE] property in its ** text encoding argument when the SQL function is ** [sqlite3_create_function|registered]. If the [SQLITE_RESULT_SUBTYPE] ** property is omitted from the function that invokes sqlite3_result_subtype(), ** then in some cases the sqlite3_result_subtype() might fail to set ** the result subtype. ** ** If SQLite is compiled with -DSQLITE_STRICT_SUBTYPE=1, then any ** SQL function that invokes the sqlite3_result_subtype() interface ** and that does not have the SQLITE_RESULT_SUBTYPE property will raise ** an error. Future versions of SQLite might enable -DSQLITE_STRICT_SUBTYPE=1 ** by default. */ SQLITE_API void sqlite3_result_subtype(sqlite3_context*,unsigned int); /* ** CAPI3REF: Define New Collating Sequences ** METHOD: sqlite3 ** |
︙ | ︙ | |||
6934 6935 6936 6937 6938 6939 6940 | ** </ol> ** ^If the S argument to sqlite3_txn_state(D,S) is not the name of ** a valid schema, then -1 is returned. */ SQLITE_API int sqlite3_txn_state(sqlite3*,const char *zSchema); /* | | | 6980 6981 6982 6983 6984 6985 6986 6987 6988 6989 6990 6991 6992 6993 6994 | ** </ol> ** ^If the S argument to sqlite3_txn_state(D,S) is not the name of ** a valid schema, then -1 is returned. */ SQLITE_API int sqlite3_txn_state(sqlite3*,const char *zSchema); /* ** CAPI3REF: Allowed return values from sqlite3_txn_state() ** KEYWORDS: {transaction state} ** ** These constants define the current transaction state of a database file. ** ^The [sqlite3_txn_state(D,S)] interface returns one of these ** constants in order to describe the transaction state of schema S ** in [database connection] D. ** |
︙ | ︙ | |||
7066 7067 7068 7069 7070 7071 7072 | ** invoked whenever the database connection closes or when the callback ** is overwritten by another invocation of sqlite3_autovacuum_pages(). ** ** <p>^There is only one autovacuum pages callback per database connection. ** ^Each call to the sqlite3_autovacuum_pages() interface overrides all ** previous invocations for that database connection. ^If the callback ** argument (C) to sqlite3_autovacuum_pages(D,C,P,X) is a NULL pointer, | | | 7112 7113 7114 7115 7116 7117 7118 7119 7120 7121 7122 7123 7124 7125 7126 | ** invoked whenever the database connection closes or when the callback ** is overwritten by another invocation of sqlite3_autovacuum_pages(). ** ** <p>^There is only one autovacuum pages callback per database connection. ** ^Each call to the sqlite3_autovacuum_pages() interface overrides all ** previous invocations for that database connection. ^If the callback ** argument (C) to sqlite3_autovacuum_pages(D,C,P,X) is a NULL pointer, ** then the autovacuum steps callback is canceled. The return value ** from sqlite3_autovacuum_pages() is normally SQLITE_OK, but might ** be some other error code if something goes wrong. The current ** implementation will only return SQLITE_OK or SQLITE_MISUSE, but other ** return codes might be added in future releases. ** ** <p>If no autovacuum pages callback is specified (the usual case) or ** a NULL pointer is provided for the callback, |
︙ | ︙ | |||
7587 7588 7589 7590 7591 7592 7593 | int (*xRelease)(sqlite3_vtab *pVTab, int); int (*xRollbackTo)(sqlite3_vtab *pVTab, int); /* The methods above are in versions 1 and 2 of the sqlite_module object. ** Those below are for version 3 and greater. */ int (*xShadowName)(const char*); /* The methods above are in versions 1 through 3 of the sqlite_module object. ** Those below are for version 4 and greater. */ | | > | 7633 7634 7635 7636 7637 7638 7639 7640 7641 7642 7643 7644 7645 7646 7647 7648 | int (*xRelease)(sqlite3_vtab *pVTab, int); int (*xRollbackTo)(sqlite3_vtab *pVTab, int); /* The methods above are in versions 1 and 2 of the sqlite_module object. ** Those below are for version 3 and greater. */ int (*xShadowName)(const char*); /* The methods above are in versions 1 through 3 of the sqlite_module object. ** Those below are for version 4 and greater. */ int (*xIntegrity)(sqlite3_vtab *pVTab, const char *zSchema, const char *zTabName, int mFlags, char **pzErr); }; /* ** CAPI3REF: Virtual Table Indexing Information ** KEYWORDS: sqlite3_index_info ** ** The sqlite3_index_info structure and its substructures is used as part |
︙ | ︙ | |||
8075 8076 8077 8078 8079 8080 8081 | ** ^If the blob handle being closed was opened for read-write access, and if ** the database is in auto-commit mode and there are no other open read-write ** blob handles or active write statements, the current transaction is ** committed. ^If an error occurs while committing the transaction, an error ** code is returned and the transaction rolled back. ** ** Calling this function with an argument that is not a NULL pointer or an | | | 8122 8123 8124 8125 8126 8127 8128 8129 8130 8131 8132 8133 8134 8135 8136 | ** ^If the blob handle being closed was opened for read-write access, and if ** the database is in auto-commit mode and there are no other open read-write ** blob handles or active write statements, the current transaction is ** committed. ^If an error occurs while committing the transaction, an error ** code is returned and the transaction rolled back. ** ** Calling this function with an argument that is not a NULL pointer or an ** open blob handle results in undefined behavior. ^Calling this routine ** with a null pointer (such as would be returned by a failed call to ** [sqlite3_blob_open()]) is a harmless no-op. ^Otherwise, if this function ** is passed a valid open blob handle, the values returned by the ** sqlite3_errcode() and sqlite3_errmsg() functions are set before returning. */ SQLITE_API int sqlite3_blob_close(sqlite3_blob *); |
︙ | ︙ | |||
8302 8303 8304 8305 8306 8307 8308 | ** In such cases, the ** mutex must be exited an equal number of times before another thread ** can enter.)^ If the same thread tries to enter any mutex other ** than an SQLITE_MUTEX_RECURSIVE more than once, the behavior is undefined. ** ** ^(Some systems (for example, Windows 95) do not support the operation ** implemented by sqlite3_mutex_try(). On those systems, sqlite3_mutex_try() | | | | > > | 8349 8350 8351 8352 8353 8354 8355 8356 8357 8358 8359 8360 8361 8362 8363 8364 8365 8366 8367 | ** In such cases, the ** mutex must be exited an equal number of times before another thread ** can enter.)^ If the same thread tries to enter any mutex other ** than an SQLITE_MUTEX_RECURSIVE more than once, the behavior is undefined. ** ** ^(Some systems (for example, Windows 95) do not support the operation ** implemented by sqlite3_mutex_try(). On those systems, sqlite3_mutex_try() ** will always return SQLITE_BUSY. In most cases the SQLite core only uses ** sqlite3_mutex_try() as an optimization, so this is acceptable ** behavior. The exceptions are unix builds that set the ** SQLITE_ENABLE_SETLK_TIMEOUT build option. In that case a working ** sqlite3_mutex_try() is required.)^ ** ** ^The sqlite3_mutex_leave() routine exits a mutex that was ** previously entered by the same thread. The behavior ** is undefined if the mutex is not currently entered by the ** calling thread or is not currently allocated. ** ** ^If the argument to sqlite3_mutex_enter(), sqlite3_mutex_try(), |
︙ | ︙ | |||
8563 8564 8565 8566 8567 8568 8569 8570 8571 8572 8573 8574 8575 8576 | #define SQLITE_TESTCTRL_BITVEC_TEST 8 #define SQLITE_TESTCTRL_FAULT_INSTALL 9 #define SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS 10 #define SQLITE_TESTCTRL_PENDING_BYTE 11 #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 /* NOT USED */ #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 /* NOT USED */ #define SQLITE_TESTCTRL_SCRATCHMALLOC 17 /* NOT USED */ #define SQLITE_TESTCTRL_INTERNAL_FUNCTIONS 17 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 18 #define SQLITE_TESTCTRL_EXPLAIN_STMT 19 /* NOT USED */ #define SQLITE_TESTCTRL_ONCE_RESET_THRESHOLD 19 | > | 8612 8613 8614 8615 8616 8617 8618 8619 8620 8621 8622 8623 8624 8625 8626 | #define SQLITE_TESTCTRL_BITVEC_TEST 8 #define SQLITE_TESTCTRL_FAULT_INSTALL 9 #define SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS 10 #define SQLITE_TESTCTRL_PENDING_BYTE 11 #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 /* NOT USED */ #define SQLITE_TESTCTRL_JSON_SELFCHECK 14 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 /* NOT USED */ #define SQLITE_TESTCTRL_SCRATCHMALLOC 17 /* NOT USED */ #define SQLITE_TESTCTRL_INTERNAL_FUNCTIONS 17 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 18 #define SQLITE_TESTCTRL_EXPLAIN_STMT 19 /* NOT USED */ #define SQLITE_TESTCTRL_ONCE_RESET_THRESHOLD 19 |
︙ | ︙ | |||
9617 9618 9619 9620 9621 9622 9623 | ** the other connections to use as the blocking connection. ** ** ^(There may be at most one unlock-notify callback registered by a ** blocked connection. If sqlite3_unlock_notify() is called when the ** blocked connection already has a registered unlock-notify callback, ** then the new callback replaces the old.)^ ^If sqlite3_unlock_notify() is ** called with a NULL pointer as its second argument, then any existing | | | | 9667 9668 9669 9670 9671 9672 9673 9674 9675 9676 9677 9678 9679 9680 9681 9682 | ** the other connections to use as the blocking connection. ** ** ^(There may be at most one unlock-notify callback registered by a ** blocked connection. If sqlite3_unlock_notify() is called when the ** blocked connection already has a registered unlock-notify callback, ** then the new callback replaces the old.)^ ^If sqlite3_unlock_notify() is ** called with a NULL pointer as its second argument, then any existing ** unlock-notify callback is canceled. ^The blocked connections ** unlock-notify callback may also be canceled by closing the blocked ** connection using [sqlite3_close()]. ** ** The unlock-notify callback is not reentrant. If an application invokes ** any sqlite3_xxx API functions from within an unlock-notify callback, a ** crash or deadlock may be the result. ** ** ^Unless deadlock is detected (see below), sqlite3_unlock_notify() always |
︙ | ︙ | |||
13076 13077 13078 13079 13080 13081 13082 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: | > > > | | > > | | | > | | | | 13126 13127 13128 13129 13130 13131 13132 13133 13134 13135 13136 13137 13138 13139 13140 13141 13142 13143 13144 13145 13146 13147 13148 13149 13150 13151 13152 13153 13154 13155 13156 13157 13158 13159 13160 13161 13162 13163 13164 13165 13166 13167 13168 13169 13170 13171 13172 13173 13174 13175 13176 13177 13178 13179 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: ** If parameter iCol is less than zero, or greater than or equal to the ** number of columns in the table, SQLITE_RANGE is returned. ** ** Otherwise, this function attempts to retrieve the text of column iCol of ** the current document. If successful, (*pz) is set to point to a buffer ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes ** (not characters) of the buffer and SQLITE_OK is returned. Otherwise, ** if an error occurs, an SQLite error code is returned and the final values ** of (*pz) and (*pn) are undefined. ** ** xPhraseCount: ** Returns the number of phrases in the current query expression. ** ** xPhraseSize: ** If parameter iCol is less than zero, or greater than or equal to the ** number of phrases in the current query, as returned by xPhraseCount, ** 0 is returned. Otherwise, this function returns the number of tokens in ** phrase iPhrase of the query. Phrases are numbered starting from zero. ** ** xInstCount: ** Set *pnInst to the total number of occurrences of all phrases within ** the query within the current row. Return SQLITE_OK if successful, or ** an error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. If the FTS5 table is created ** with either "detail=none" or "detail=column" and "content=" option ** (i.e. if it is a contentless table), then this API always returns 0. ** ** xInst: ** Query for the details of phrase match iIdx within the current row. ** Phrase matches are numbered starting from zero, so the iIdx argument ** should be greater than or equal to zero and smaller than the value ** output by xInstCount(). If iIdx is less than zero or greater than ** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned. ** ** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol ** to the column in which it occurs and *piOff the token offset of the ** first token of the phrase. SQLITE_OK is returned if successful, or an ** error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. ** ** xRowid: ** Returns the rowid of the current row. ** |
︙ | ︙ | |||
13134 13135 13136 13137 13138 13139 13140 13141 13142 13143 13144 13145 13146 13147 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. | > > > > | 13190 13191 13192 13193 13194 13195 13196 13197 13198 13199 13200 13201 13202 13203 13204 13205 13206 13207 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If parameter iPhrase is less than zero, or greater than or equal to ** the number of phrases in the query, as returned by xPhraseCount(), ** this function returns SQLITE_RANGE. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. |
︙ | ︙ | |||
13249 13250 13251 13252 13253 13254 13255 13256 13257 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. */ struct Fts5ExtensionApi { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | 13309 13310 13311 13312 13313 13314 13315 13316 13317 13318 13319 13320 13321 13322 13323 13324 13325 13326 13327 13328 13329 13330 13331 13332 13333 13334 13335 13336 13337 13338 13339 13340 13341 13342 13343 13344 13345 13346 13347 13348 13349 13350 13351 13352 13353 13354 13355 13356 13357 13358 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. ** ** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase iPhrase of the current ** query. Before returning, output parameter *ppToken is set to point ** to a buffer containing the requested token, and *pnToken to the ** size of this buffer in bytes. ** ** If iPhrase or iToken are less than zero, or if iPhrase is greater than ** or equal to the number of phrases in the query as reported by ** xPhraseCount(), or if iToken is equal to or greater than the number of ** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken are both zeroed. ** ** The output text is not a copy of the query text that specified the ** token. It is the output of the tokenizer module. For tokendata=1 ** tables, this includes any embedded 0x00 and trailing data. ** ** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase hit iIdx within the ** current row. If iIdx is less than zero or greater than or equal to the ** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise, ** output variable (*ppToken) is set to point to a buffer containing the ** matching document token, and (*pnToken) to the size of that buffer in ** bytes. This API is not available if the specified token matches a ** prefix query term. In that case both output variables are always set ** to 0. ** ** The output text is not a copy of the document text that was tokenized. ** It is the output of the tokenizer module. For tokendata=1 tables, this ** includes any embedded 0x00 and trailing data. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. */ struct Fts5ExtensionApi { int iVersion; /* Currently always set to 3 */ void *(*xUserData)(Fts5Context*); int (*xColumnCount)(Fts5Context*); int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow); int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken); |
︙ | ︙ | |||
13286 13287 13288 13289 13290 13291 13292 13293 13294 13295 13296 13297 13298 13299 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* | > > > > > > > | 13379 13380 13381 13382 13383 13384 13385 13386 13387 13388 13389 13390 13391 13392 13393 13394 13395 13396 13397 13398 13399 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); /* Below this point are iVersion>=3 only */ int (*xQueryToken)(Fts5Context*, int iPhrase, int iToken, const char **ppToken, int *pnToken ); int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* |
︙ | ︙ | |||
13772 13773 13774 13775 13776 13777 13778 | ** Maximum number of pages in one database file. ** ** This is really just the default value for the max_page_count pragma. ** This value can be lowered (or raised) at run-time using that the ** max_page_count macro. */ #ifndef SQLITE_MAX_PAGE_COUNT | | | 13872 13873 13874 13875 13876 13877 13878 13879 13880 13881 13882 13883 13884 13885 13886 | ** Maximum number of pages in one database file. ** ** This is really just the default value for the max_page_count pragma. ** This value can be lowered (or raised) at run-time using that the ** max_page_count macro. */ #ifndef SQLITE_MAX_PAGE_COUNT # define SQLITE_MAX_PAGE_COUNT 0xfffffffe /* 4294967294 */ #endif /* ** Maximum length (in bytes) of the pattern in a LIKE or GLOB ** operator. */ #ifndef SQLITE_MAX_LIKE_PATTERN_LENGTH |
︙ | ︙ | |||
13910 13911 13912 13913 13914 13915 13916 13917 13918 13919 13920 13921 13922 13923 | ** SEH support if the -DSQLITE_OMIT_SEH option is given. */ #if defined(_MSC_VER) && !defined(SQLITE_OMIT_SEH) # define SQLITE_USE_SEH 1 #else # undef SQLITE_USE_SEH #endif /* ** The SQLITE_THREADSAFE macro must be defined as 0, 1, or 2. ** 0 means mutexes are permanently disable and the library is never ** threadsafe. 1 means the library is serialized which is the highest ** level of threadsafety. 2 means the library is multithreaded - multiple ** threads can use SQLite as long as no two threads try to use the same | > > > > > > > > > > > > > | 14010 14011 14012 14013 14014 14015 14016 14017 14018 14019 14020 14021 14022 14023 14024 14025 14026 14027 14028 14029 14030 14031 14032 14033 14034 14035 14036 | ** SEH support if the -DSQLITE_OMIT_SEH option is given. */ #if defined(_MSC_VER) && !defined(SQLITE_OMIT_SEH) # define SQLITE_USE_SEH 1 #else # undef SQLITE_USE_SEH #endif /* ** Enable SQLITE_DIRECT_OVERFLOW_READ, unless the build explicitly ** disables it using -DSQLITE_DIRECT_OVERFLOW_READ=0 */ #if defined(SQLITE_DIRECT_OVERFLOW_READ) && SQLITE_DIRECT_OVERFLOW_READ+1==1 /* Disable if -DSQLITE_DIRECT_OVERFLOW_READ=0 */ # undef SQLITE_DIRECT_OVERFLOW_READ #else /* In all other cases, enable */ # define SQLITE_DIRECT_OVERFLOW_READ 1 #endif /* ** The SQLITE_THREADSAFE macro must be defined as 0, 1, or 2. ** 0 means mutexes are permanently disable and the library is never ** threadsafe. 1 means the library is serialized which is the highest ** level of threadsafety. 2 means the library is multithreaded - multiple ** threads can use SQLite as long as no two threads try to use the same |
︙ | ︙ | |||
15793 15794 15795 15796 15797 15798 15799 | SQLITE_PRIVATE const char *sqlite3PagerFilename(const Pager*, int); SQLITE_PRIVATE sqlite3_vfs *sqlite3PagerVfs(Pager*); SQLITE_PRIVATE sqlite3_file *sqlite3PagerFile(Pager*); SQLITE_PRIVATE sqlite3_file *sqlite3PagerJrnlFile(Pager*); SQLITE_PRIVATE const char *sqlite3PagerJournalname(Pager*); SQLITE_PRIVATE void *sqlite3PagerTempSpace(Pager*); SQLITE_PRIVATE int sqlite3PagerIsMemdb(Pager*); | | | 15906 15907 15908 15909 15910 15911 15912 15913 15914 15915 15916 15917 15918 15919 15920 | SQLITE_PRIVATE const char *sqlite3PagerFilename(const Pager*, int); SQLITE_PRIVATE sqlite3_vfs *sqlite3PagerVfs(Pager*); SQLITE_PRIVATE sqlite3_file *sqlite3PagerFile(Pager*); SQLITE_PRIVATE sqlite3_file *sqlite3PagerJrnlFile(Pager*); SQLITE_PRIVATE const char *sqlite3PagerJournalname(Pager*); SQLITE_PRIVATE void *sqlite3PagerTempSpace(Pager*); SQLITE_PRIVATE int sqlite3PagerIsMemdb(Pager*); SQLITE_PRIVATE void sqlite3PagerCacheStat(Pager *, int, int, u64*); SQLITE_PRIVATE void sqlite3PagerClearCache(Pager*); SQLITE_PRIVATE int sqlite3SectorSize(sqlite3_file *); /* Functions used to truncate the database file. */ SQLITE_PRIVATE void sqlite3PagerTruncateImage(Pager*,Pgno); SQLITE_PRIVATE void sqlite3PagerRekey(DbPage*, Pgno, u16); |
︙ | ︙ | |||
16380 16381 16382 16383 16384 16385 16386 16387 16388 16389 16390 16391 16392 16393 | #define P4_EXPR (-9) /* P4 is a pointer to an Expr tree */ #define P4_MEM (-10) /* P4 is a pointer to a Mem* structure */ #define P4_VTAB (-11) /* P4 is a pointer to an sqlite3_vtab structure */ #define P4_REAL (-12) /* P4 is a 64-bit floating point value */ #define P4_INT64 (-13) /* P4 is a 64-bit signed integer */ #define P4_INTARRAY (-14) /* P4 is a vector of 32-bit integers */ #define P4_FUNCCTX (-15) /* P4 is a pointer to an sqlite3_context object */ /* Error message codes for OP_Halt */ #define P5_ConstraintNotNull 1 #define P5_ConstraintUnique 2 #define P5_ConstraintCheck 3 #define P5_ConstraintFK 4 | > | 16493 16494 16495 16496 16497 16498 16499 16500 16501 16502 16503 16504 16505 16506 16507 | #define P4_EXPR (-9) /* P4 is a pointer to an Expr tree */ #define P4_MEM (-10) /* P4 is a pointer to a Mem* structure */ #define P4_VTAB (-11) /* P4 is a pointer to an sqlite3_vtab structure */ #define P4_REAL (-12) /* P4 is a 64-bit floating point value */ #define P4_INT64 (-13) /* P4 is a 64-bit signed integer */ #define P4_INTARRAY (-14) /* P4 is a vector of 32-bit integers */ #define P4_FUNCCTX (-15) /* P4 is a pointer to an sqlite3_context object */ #define P4_TABLEREF (-16) /* Like P4_TABLE, but reference counted */ /* Error message codes for OP_Halt */ #define P5_ConstraintNotNull 1 #define P5_ConstraintUnique 2 #define P5_ConstraintCheck 3 #define P5_ConstraintFK 4 |
︙ | ︙ | |||
16602 16603 16604 16605 16606 16607 16608 | #define OP_VCheck 174 #define OP_VInitIn 175 /* synopsis: r[P2]=ValueList(P1,P3) */ #define OP_VColumn 176 /* synopsis: r[P3]=vcolumn(P2) */ #define OP_VRename 177 #define OP_Pagecount 178 #define OP_MaxPgcnt 179 #define OP_ClrSubtype 180 /* synopsis: r[P1].subtype = 0 */ | > > | | | | | | | | 16716 16717 16718 16719 16720 16721 16722 16723 16724 16725 16726 16727 16728 16729 16730 16731 16732 16733 16734 16735 16736 16737 16738 | #define OP_VCheck 174 #define OP_VInitIn 175 /* synopsis: r[P2]=ValueList(P1,P3) */ #define OP_VColumn 176 /* synopsis: r[P3]=vcolumn(P2) */ #define OP_VRename 177 #define OP_Pagecount 178 #define OP_MaxPgcnt 179 #define OP_ClrSubtype 180 /* synopsis: r[P1].subtype = 0 */ #define OP_GetSubtype 181 /* synopsis: r[P2] = r[P1].subtype */ #define OP_SetSubtype 182 /* synopsis: r[P2].subtype = r[P1] */ #define OP_FilterAdd 183 /* synopsis: filter(P1) += key(P3@P4) */ #define OP_Trace 184 #define OP_CursorHint 185 #define OP_ReleaseReg 186 /* synopsis: release r[P1@P2] mask P3 */ #define OP_Noop 187 #define OP_Explain 188 #define OP_Abortable 189 /* Properties such as "out2" or "jump" that are specified in ** comments following the "case" for each opcode in the vdbe.c ** are encoded into bitvectors as follows: */ #define OPFLG_JUMP 0x01 /* jump: P2 holds jmp target */ #define OPFLG_IN1 0x02 /* in1: P1 is an input */ |
︙ | ︙ | |||
16644 16645 16646 16647 16648 16649 16650 | /* 120 */ 0x00, 0x00, 0x40, 0x00, 0x40, 0x40, 0x10, 0x10,\ /* 128 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x50,\ /* 136 */ 0x00, 0x40, 0x04, 0x04, 0x00, 0x40, 0x50, 0x40,\ /* 144 */ 0x10, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00,\ /* 152 */ 0x00, 0x10, 0x00, 0x00, 0x06, 0x10, 0x00, 0x04,\ /* 160 */ 0x1a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\ /* 168 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x10, 0x50,\ | | | | 16760 16761 16762 16763 16764 16765 16766 16767 16768 16769 16770 16771 16772 16773 16774 16775 | /* 120 */ 0x00, 0x00, 0x40, 0x00, 0x40, 0x40, 0x10, 0x10,\ /* 128 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x00, 0x50,\ /* 136 */ 0x00, 0x40, 0x04, 0x04, 0x00, 0x40, 0x50, 0x40,\ /* 144 */ 0x10, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00,\ /* 152 */ 0x00, 0x10, 0x00, 0x00, 0x06, 0x10, 0x00, 0x04,\ /* 160 */ 0x1a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,\ /* 168 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x10, 0x50,\ /* 176 */ 0x40, 0x00, 0x10, 0x10, 0x02, 0x12, 0x12, 0x00,\ /* 184 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,} /* The resolve3P2Values() routine is able to run faster if it knows ** the value of the largest JUMP opcode. The smaller the maximum ** JUMP opcode the better, so the mkopcodeh.tcl script that ** generated this include file strives to group all JUMP opcodes ** together near the beginning of the list. */ |
︙ | ︙ | |||
17806 17807 17808 17809 17810 17811 17812 | /* 0x0200 -- available for reuse */ #define SQLITE_FUNC_UNLIKELY 0x0400 /* Built-in unlikely() function */ #define SQLITE_FUNC_CONSTANT 0x0800 /* Constant inputs give a constant output */ #define SQLITE_FUNC_MINMAX 0x1000 /* True for min() and max() aggregates */ #define SQLITE_FUNC_SLOCHNG 0x2000 /* "Slow Change". Value constant during a ** single query - might change over time */ #define SQLITE_FUNC_TEST 0x4000 /* Built-in testing functions */ | | | > | 17922 17923 17924 17925 17926 17927 17928 17929 17930 17931 17932 17933 17934 17935 17936 17937 17938 17939 17940 17941 17942 17943 17944 | /* 0x0200 -- available for reuse */ #define SQLITE_FUNC_UNLIKELY 0x0400 /* Built-in unlikely() function */ #define SQLITE_FUNC_CONSTANT 0x0800 /* Constant inputs give a constant output */ #define SQLITE_FUNC_MINMAX 0x1000 /* True for min() and max() aggregates */ #define SQLITE_FUNC_SLOCHNG 0x2000 /* "Slow Change". Value constant during a ** single query - might change over time */ #define SQLITE_FUNC_TEST 0x4000 /* Built-in testing functions */ #define SQLITE_FUNC_RUNONLY 0x8000 /* Cannot be used by valueFromFunction */ #define SQLITE_FUNC_WINDOW 0x00010000 /* Built-in window-only function */ #define SQLITE_FUNC_INTERNAL 0x00040000 /* For use by NestedParse() only */ #define SQLITE_FUNC_DIRECT 0x00080000 /* Not for use in TRIGGERs or VIEWs */ /* SQLITE_SUBTYPE 0x00100000 // Consumer of subtypes */ #define SQLITE_FUNC_UNSAFE 0x00200000 /* Function has side effects */ #define SQLITE_FUNC_INLINE 0x00400000 /* Functions implemented in-line */ #define SQLITE_FUNC_BUILTIN 0x00800000 /* This is a built-in function */ /* SQLITE_RESULT_SUBTYPE 0x01000000 // Generator of subtypes */ #define SQLITE_FUNC_ANYORDER 0x08000000 /* count/min/max aggregate */ /* Identifier numbers for each in-line function */ #define INLINEFUNC_coalesce 0 #define INLINEFUNC_implies_nonnull_row 1 #define INLINEFUNC_expr_implies_expr 2 #define INLINEFUNC_expr_compare 3 |
︙ | ︙ | |||
17905 17906 17907 17908 17909 17910 17911 | SQLITE_INT_TO_PTR(iArg), 0, xFunc, 0, 0, 0, #zName, {0} } #define SFUNCTION(zName, nArg, iArg, bNC, xFunc) \ {nArg, SQLITE_FUNC_BUILTIN|SQLITE_UTF8|SQLITE_DIRECTONLY|SQLITE_FUNC_UNSAFE, \ SQLITE_INT_TO_PTR(iArg), 0, xFunc, 0, 0, 0, #zName, {0} } #define MFUNCTION(zName, nArg, xPtr, xFunc) \ {nArg, SQLITE_FUNC_BUILTIN|SQLITE_FUNC_CONSTANT|SQLITE_UTF8, \ xPtr, 0, xFunc, 0, 0, 0, #zName, {0} } | | | | > | | 18022 18023 18024 18025 18026 18027 18028 18029 18030 18031 18032 18033 18034 18035 18036 18037 18038 18039 18040 | SQLITE_INT_TO_PTR(iArg), 0, xFunc, 0, 0, 0, #zName, {0} } #define SFUNCTION(zName, nArg, iArg, bNC, xFunc) \ {nArg, SQLITE_FUNC_BUILTIN|SQLITE_UTF8|SQLITE_DIRECTONLY|SQLITE_FUNC_UNSAFE, \ SQLITE_INT_TO_PTR(iArg), 0, xFunc, 0, 0, 0, #zName, {0} } #define MFUNCTION(zName, nArg, xPtr, xFunc) \ {nArg, SQLITE_FUNC_BUILTIN|SQLITE_FUNC_CONSTANT|SQLITE_UTF8, \ xPtr, 0, xFunc, 0, 0, 0, #zName, {0} } #define JFUNCTION(zName, nArg, bUseCache, bWS, bRS, bJsonB, iArg, xFunc) \ {nArg, SQLITE_FUNC_BUILTIN|SQLITE_DETERMINISTIC|SQLITE_FUNC_CONSTANT|\ SQLITE_UTF8|((bUseCache)*SQLITE_FUNC_RUNONLY)|\ ((bRS)*SQLITE_SUBTYPE)|((bWS)*SQLITE_RESULT_SUBTYPE), \ SQLITE_INT_TO_PTR(iArg|((bJsonB)*JSON_BLOB)),0,xFunc,0, 0, 0, #zName, {0} } #define INLINE_FUNC(zName, nArg, iArg, mFlags) \ {nArg, SQLITE_FUNC_BUILTIN|\ SQLITE_UTF8|SQLITE_FUNC_INLINE|SQLITE_FUNC_CONSTANT|(mFlags), \ SQLITE_INT_TO_PTR(iArg), 0, noopFunc, 0, 0, 0, #zName, {0} } #define TEST_FUNC(zName, nArg, iArg, mFlags) \ {nArg, SQLITE_FUNC_BUILTIN|\ SQLITE_UTF8|SQLITE_FUNC_INTERNAL|SQLITE_FUNC_TEST| \ |
︙ | ︙ | |||
18543 18544 18545 18546 18547 18548 18549 18550 18551 18552 18553 18554 18555 18556 | unsigned idxType:2; /* 0:Normal 1:UNIQUE, 2:PRIMARY KEY, 3:IPK */ unsigned bUnordered:1; /* Use this index for == or IN queries only */ unsigned uniqNotNull:1; /* True if UNIQUE and NOT NULL for all columns */ unsigned isResized:1; /* True if resizeIndexObject() has been called */ unsigned isCovering:1; /* True if this is a covering index */ unsigned noSkipScan:1; /* Do not try to use skip-scan if true */ unsigned hasStat1:1; /* aiRowLogEst values come from sqlite_stat1 */ unsigned bNoQuery:1; /* Do not use this index to optimize queries */ unsigned bAscKeyBug:1; /* True if the bba7b69f9849b5bf bug applies */ unsigned bHasVCol:1; /* Index references one or more VIRTUAL columns */ unsigned bHasExpr:1; /* Index contains an expression, either a literal ** expression, or a reference to a VIRTUAL column */ #ifdef SQLITE_ENABLE_STAT4 int nSample; /* Number of elements in aSample[] */ | > | 18661 18662 18663 18664 18665 18666 18667 18668 18669 18670 18671 18672 18673 18674 18675 | unsigned idxType:2; /* 0:Normal 1:UNIQUE, 2:PRIMARY KEY, 3:IPK */ unsigned bUnordered:1; /* Use this index for == or IN queries only */ unsigned uniqNotNull:1; /* True if UNIQUE and NOT NULL for all columns */ unsigned isResized:1; /* True if resizeIndexObject() has been called */ unsigned isCovering:1; /* True if this is a covering index */ unsigned noSkipScan:1; /* Do not try to use skip-scan if true */ unsigned hasStat1:1; /* aiRowLogEst values come from sqlite_stat1 */ unsigned bLowQual:1; /* sqlite_stat1 says this is a low-quality index */ unsigned bNoQuery:1; /* Do not use this index to optimize queries */ unsigned bAscKeyBug:1; /* True if the bba7b69f9849b5bf bug applies */ unsigned bHasVCol:1; /* Index references one or more VIRTUAL columns */ unsigned bHasExpr:1; /* Index contains an expression, either a literal ** expression, or a reference to a VIRTUAL column */ #ifdef SQLITE_ENABLE_STAT4 int nSample; /* Number of elements in aSample[] */ |
︙ | ︙ | |||
18656 18657 18658 18659 18660 18661 18662 18663 18664 18665 18666 18667 18668 18669 | Expr *pFExpr; /* Expression encoding the function */ FuncDef *pFunc; /* The aggregate function implementation */ int iDistinct; /* Ephemeral table used to enforce DISTINCT */ int iDistAddr; /* Address of OP_OpenEphemeral */ int iOBTab; /* Ephemeral table to implement ORDER BY */ u8 bOBPayload; /* iOBTab has payload columns separate from key */ u8 bOBUnique; /* Enforce uniqueness on iOBTab keys */ } *aFunc; int nFunc; /* Number of entries in aFunc[] */ u32 selId; /* Select to which this AggInfo belongs */ #ifdef SQLITE_DEBUG Select *pSelect; /* SELECT statement that this AggInfo supports */ #endif }; | > | 18775 18776 18777 18778 18779 18780 18781 18782 18783 18784 18785 18786 18787 18788 18789 | Expr *pFExpr; /* Expression encoding the function */ FuncDef *pFunc; /* The aggregate function implementation */ int iDistinct; /* Ephemeral table used to enforce DISTINCT */ int iDistAddr; /* Address of OP_OpenEphemeral */ int iOBTab; /* Ephemeral table to implement ORDER BY */ u8 bOBPayload; /* iOBTab has payload columns separate from key */ u8 bOBUnique; /* Enforce uniqueness on iOBTab keys */ u8 bUseSubtype; /* Transfer subtype info through sorter */ } *aFunc; int nFunc; /* Number of entries in aFunc[] */ u32 selId; /* Select to which this AggInfo belongs */ #ifdef SQLITE_DEBUG Select *pSelect; /* SELECT statement that this AggInfo supports */ #endif }; |
︙ | ︙ | |||
19189 19190 19191 19192 19193 19194 19195 19196 19197 19198 19199 19200 19201 19202 | Upsert *pUpsert; /* ON CONFLICT clause information from an upsert */ int iBaseReg; /* For TK_REGISTER when parsing RETURNING */ } uNC; NameContext *pNext; /* Next outer name context. NULL for outermost */ int nRef; /* Number of names resolved by this context */ int nNcErr; /* Number of errors encountered while resolving names */ int ncFlags; /* Zero or more NC_* flags defined below */ Select *pWinSelect; /* SELECT statement for any window functions */ }; /* ** Allowed values for the NameContext, ncFlags field. ** ** Value constraints (all checked via assert()): | > | 19309 19310 19311 19312 19313 19314 19315 19316 19317 19318 19319 19320 19321 19322 19323 | Upsert *pUpsert; /* ON CONFLICT clause information from an upsert */ int iBaseReg; /* For TK_REGISTER when parsing RETURNING */ } uNC; NameContext *pNext; /* Next outer name context. NULL for outermost */ int nRef; /* Number of names resolved by this context */ int nNcErr; /* Number of errors encountered while resolving names */ int ncFlags; /* Zero or more NC_* flags defined below */ u32 nNestedSelect; /* Number of nested selects using this NC */ Select *pWinSelect; /* SELECT statement for any window functions */ }; /* ** Allowed values for the NameContext, ncFlags field. ** ** Value constraints (all checked via assert()): |
︙ | ︙ | |||
19869 19870 19871 19872 19873 19874 19875 19876 19877 19878 19879 19880 19881 19882 | Parse *pParse; /* The parse that includes the RETURNING clause */ ExprList *pReturnEL; /* List of expressions to return */ Trigger retTrig; /* The transient trigger that implements RETURNING */ TriggerStep retTStep; /* The trigger step */ int iRetCur; /* Transient table holding RETURNING results */ int nRetCol; /* Number of in pReturnEL after expansion */ int iRetReg; /* Register array for holding a row of RETURNING */ }; /* ** An objected used to accumulate the text of a string where we ** do not necessarily know how big the string will be in the end. */ struct sqlite3_str { | > | 19990 19991 19992 19993 19994 19995 19996 19997 19998 19999 20000 20001 20002 20003 20004 | Parse *pParse; /* The parse that includes the RETURNING clause */ ExprList *pReturnEL; /* List of expressions to return */ Trigger retTrig; /* The transient trigger that implements RETURNING */ TriggerStep retTStep; /* The trigger step */ int iRetCur; /* Transient table holding RETURNING results */ int nRetCol; /* Number of in pReturnEL after expansion */ int iRetReg; /* Register array for holding a row of RETURNING */ char zName[40]; /* Name of trigger: "sqlite_returning_%p" */ }; /* ** An objected used to accumulate the text of a string where we ** do not necessarily know how big the string will be in the end. */ struct sqlite3_str { |
︙ | ︙ | |||
19904 19905 19906 19907 19908 19909 19910 19911 19912 19913 19914 19915 19916 19917 | ** when the reference count reaches zero. ** ** 2. Use sqlite3RCStrUnref() to free an RCStr string rather than ** sqlite3_free() ** ** 3. Make a (read-only) copy of a read-only RCStr string using ** sqlite3RCStrRef(). */ struct RCStr { u64 nRCRef; /* Number of references */ /* Total structure size should be a multiple of 8 bytes for alignment */ }; /* | > > > | 20026 20027 20028 20029 20030 20031 20032 20033 20034 20035 20036 20037 20038 20039 20040 20041 20042 | ** when the reference count reaches zero. ** ** 2. Use sqlite3RCStrUnref() to free an RCStr string rather than ** sqlite3_free() ** ** 3. Make a (read-only) copy of a read-only RCStr string using ** sqlite3RCStrRef(). ** ** "String" is in the name, but an RCStr object can also be used to hold ** binary data. */ struct RCStr { u64 nRCRef; /* Number of references */ /* Total structure size should be a multiple of 8 bytes for alignment */ }; /* |
︙ | ︙ | |||
19962 19963 19964 19965 19966 19967 19968 19969 19970 19971 19972 19973 19974 19975 | u8 bCoreMutex; /* True to enable core mutexing */ u8 bFullMutex; /* True to enable full mutexing */ u8 bOpenUri; /* True to interpret filenames as URIs */ u8 bUseCis; /* Use covering indices for full-scans */ u8 bSmallMalloc; /* Avoid large memory allocations if true */ u8 bExtraSchemaChecks; /* Verify type,name,tbl_name in schema */ u8 bUseLongDouble; /* Make use of long double */ int mxStrlen; /* Maximum string length */ int neverCorrupt; /* Database is always well-formed */ int szLookaside; /* Default lookaside buffer size */ int nLookaside; /* Default lookaside buffer count */ int nStmtSpill; /* Stmt-journal spill-to-disk threshold */ sqlite3_mem_methods m; /* Low-level memory allocation interface */ sqlite3_mutex_methods mutex; /* Low-level mutex interface */ | > > > | 20087 20088 20089 20090 20091 20092 20093 20094 20095 20096 20097 20098 20099 20100 20101 20102 20103 | u8 bCoreMutex; /* True to enable core mutexing */ u8 bFullMutex; /* True to enable full mutexing */ u8 bOpenUri; /* True to interpret filenames as URIs */ u8 bUseCis; /* Use covering indices for full-scans */ u8 bSmallMalloc; /* Avoid large memory allocations if true */ u8 bExtraSchemaChecks; /* Verify type,name,tbl_name in schema */ u8 bUseLongDouble; /* Make use of long double */ #ifdef SQLITE_DEBUG u8 bJsonSelfcheck; /* Double-check JSON parsing */ #endif int mxStrlen; /* Maximum string length */ int neverCorrupt; /* Database is always well-formed */ int szLookaside; /* Default lookaside buffer size */ int nLookaside; /* Default lookaside buffer count */ int nStmtSpill; /* Stmt-journal spill-to-disk threshold */ sqlite3_mem_methods m; /* Low-level memory allocation interface */ sqlite3_mutex_methods mutex; /* Low-level mutex interface */ |
︙ | ︙ | |||
20588 20589 20590 20591 20592 20593 20594 20595 20596 20597 20598 20599 20600 20601 20602 20603 20604 20605 20606 20607 20608 20609 20610 | SQLITE_PRIVATE Expr *sqlite3ExprSimplifiedAndOr(Expr*); SQLITE_PRIVATE Expr *sqlite3ExprFunction(Parse*,ExprList*, const Token*, int); SQLITE_PRIVATE void sqlite3ExprAddFunctionOrderBy(Parse*,Expr*,ExprList*); SQLITE_PRIVATE void sqlite3ExprOrderByAggregateError(Parse*,Expr*); SQLITE_PRIVATE void sqlite3ExprFunctionUsable(Parse*,const Expr*,const FuncDef*); SQLITE_PRIVATE void sqlite3ExprAssignVarNumber(Parse*, Expr*, u32); SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3*, Expr*); SQLITE_PRIVATE void sqlite3ExprDeferredDelete(Parse*, Expr*); SQLITE_PRIVATE void sqlite3ExprUnmapAndDelete(Parse*, Expr*); SQLITE_PRIVATE ExprList *sqlite3ExprListAppend(Parse*,ExprList*,Expr*); SQLITE_PRIVATE ExprList *sqlite3ExprListAppendVector(Parse*,ExprList*,IdList*,Expr*); SQLITE_PRIVATE Select *sqlite3ExprListToValues(Parse*, int, ExprList*); SQLITE_PRIVATE void sqlite3ExprListSetSortOrder(ExprList*,int,int); SQLITE_PRIVATE void sqlite3ExprListSetName(Parse*,ExprList*,const Token*,int); SQLITE_PRIVATE void sqlite3ExprListSetSpan(Parse*,ExprList*,const char*,const char*); SQLITE_PRIVATE void sqlite3ExprListDelete(sqlite3*, ExprList*); SQLITE_PRIVATE u32 sqlite3ExprListFlags(const ExprList*); SQLITE_PRIVATE int sqlite3IndexHasDuplicateRootPage(Index*); SQLITE_PRIVATE int sqlite3Init(sqlite3*, char**); SQLITE_PRIVATE int sqlite3InitCallback(void*, int, char**, char**); SQLITE_PRIVATE int sqlite3InitOne(sqlite3*, int, char**, u32); SQLITE_PRIVATE void sqlite3Pragma(Parse*,Token*,Token*,Token*,int); #ifndef SQLITE_OMIT_VIRTUALTABLE | > > | 20716 20717 20718 20719 20720 20721 20722 20723 20724 20725 20726 20727 20728 20729 20730 20731 20732 20733 20734 20735 20736 20737 20738 20739 20740 | SQLITE_PRIVATE Expr *sqlite3ExprSimplifiedAndOr(Expr*); SQLITE_PRIVATE Expr *sqlite3ExprFunction(Parse*,ExprList*, const Token*, int); SQLITE_PRIVATE void sqlite3ExprAddFunctionOrderBy(Parse*,Expr*,ExprList*); SQLITE_PRIVATE void sqlite3ExprOrderByAggregateError(Parse*,Expr*); SQLITE_PRIVATE void sqlite3ExprFunctionUsable(Parse*,const Expr*,const FuncDef*); SQLITE_PRIVATE void sqlite3ExprAssignVarNumber(Parse*, Expr*, u32); SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3*, Expr*); SQLITE_PRIVATE void sqlite3ExprDeleteGeneric(sqlite3*,void*); SQLITE_PRIVATE void sqlite3ExprDeferredDelete(Parse*, Expr*); SQLITE_PRIVATE void sqlite3ExprUnmapAndDelete(Parse*, Expr*); SQLITE_PRIVATE ExprList *sqlite3ExprListAppend(Parse*,ExprList*,Expr*); SQLITE_PRIVATE ExprList *sqlite3ExprListAppendVector(Parse*,ExprList*,IdList*,Expr*); SQLITE_PRIVATE Select *sqlite3ExprListToValues(Parse*, int, ExprList*); SQLITE_PRIVATE void sqlite3ExprListSetSortOrder(ExprList*,int,int); SQLITE_PRIVATE void sqlite3ExprListSetName(Parse*,ExprList*,const Token*,int); SQLITE_PRIVATE void sqlite3ExprListSetSpan(Parse*,ExprList*,const char*,const char*); SQLITE_PRIVATE void sqlite3ExprListDelete(sqlite3*, ExprList*); SQLITE_PRIVATE void sqlite3ExprListDeleteGeneric(sqlite3*,void*); SQLITE_PRIVATE u32 sqlite3ExprListFlags(const ExprList*); SQLITE_PRIVATE int sqlite3IndexHasDuplicateRootPage(Index*); SQLITE_PRIVATE int sqlite3Init(sqlite3*, char**); SQLITE_PRIVATE int sqlite3InitCallback(void*, int, char**, char**); SQLITE_PRIVATE int sqlite3InitOne(sqlite3*, int, char**, u32); SQLITE_PRIVATE void sqlite3Pragma(Parse*,Token*,Token*,Token*,int); #ifndef SQLITE_OMIT_VIRTUALTABLE |
︙ | ︙ | |||
20687 20688 20689 20690 20691 20692 20693 20694 20695 20696 20697 20698 20699 20700 | #if SQLITE_MAX_ATTACHED>30 SQLITE_PRIVATE int sqlite3DbMaskAllZero(yDbMask); #endif SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int); SQLITE_PRIVATE void sqlite3CodeDropTable(Parse*, Table*, int, int); SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*); SQLITE_PRIVATE void sqlite3FreeIndex(sqlite3*, Index*); #ifndef SQLITE_OMIT_AUTOINCREMENT SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse); SQLITE_PRIVATE void sqlite3AutoincrementEnd(Parse *pParse); #else # define sqlite3AutoincrementBegin(X) # define sqlite3AutoincrementEnd(X) | > | 20817 20818 20819 20820 20821 20822 20823 20824 20825 20826 20827 20828 20829 20830 20831 | #if SQLITE_MAX_ATTACHED>30 SQLITE_PRIVATE int sqlite3DbMaskAllZero(yDbMask); #endif SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int); SQLITE_PRIVATE void sqlite3CodeDropTable(Parse*, Table*, int, int); SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*); SQLITE_PRIVATE void sqlite3DeleteTableGeneric(sqlite3*, void*); SQLITE_PRIVATE void sqlite3FreeIndex(sqlite3*, Index*); #ifndef SQLITE_OMIT_AUTOINCREMENT SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse); SQLITE_PRIVATE void sqlite3AutoincrementEnd(Parse *pParse); #else # define sqlite3AutoincrementBegin(X) # define sqlite3AutoincrementEnd(X) |
︙ | ︙ | |||
20723 20724 20725 20726 20727 20728 20729 20730 20731 20732 20733 20734 20735 20736 | SQLITE_PRIVATE void sqlite3CreateIndex(Parse*,Token*,Token*,SrcList*,ExprList*,int,Token*, Expr*, int, int, u8); SQLITE_PRIVATE void sqlite3DropIndex(Parse*, SrcList*, int); SQLITE_PRIVATE int sqlite3Select(Parse*, Select*, SelectDest*); SQLITE_PRIVATE Select *sqlite3SelectNew(Parse*,ExprList*,SrcList*,Expr*,ExprList*, Expr*,ExprList*,u32,Expr*); SQLITE_PRIVATE void sqlite3SelectDelete(sqlite3*, Select*); SQLITE_PRIVATE Table *sqlite3SrcListLookup(Parse*, SrcList*); SQLITE_PRIVATE int sqlite3IsReadOnly(Parse*, Table*, Trigger*); SQLITE_PRIVATE void sqlite3OpenTable(Parse*, int iCur, int iDb, Table*, int); #if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) && !defined(SQLITE_OMIT_SUBQUERY) SQLITE_PRIVATE Expr *sqlite3LimitWhere(Parse*,SrcList*,Expr*,ExprList*,Expr*,char*); #endif SQLITE_PRIVATE void sqlite3CodeChangeCount(Vdbe*,int,const char*); | > | 20854 20855 20856 20857 20858 20859 20860 20861 20862 20863 20864 20865 20866 20867 20868 | SQLITE_PRIVATE void sqlite3CreateIndex(Parse*,Token*,Token*,SrcList*,ExprList*,int,Token*, Expr*, int, int, u8); SQLITE_PRIVATE void sqlite3DropIndex(Parse*, SrcList*, int); SQLITE_PRIVATE int sqlite3Select(Parse*, Select*, SelectDest*); SQLITE_PRIVATE Select *sqlite3SelectNew(Parse*,ExprList*,SrcList*,Expr*,ExprList*, Expr*,ExprList*,u32,Expr*); SQLITE_PRIVATE void sqlite3SelectDelete(sqlite3*, Select*); SQLITE_PRIVATE void sqlite3SelectDeleteGeneric(sqlite3*,void*); SQLITE_PRIVATE Table *sqlite3SrcListLookup(Parse*, SrcList*); SQLITE_PRIVATE int sqlite3IsReadOnly(Parse*, Table*, Trigger*); SQLITE_PRIVATE void sqlite3OpenTable(Parse*, int iCur, int iDb, Table*, int); #if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) && !defined(SQLITE_OMIT_SUBQUERY) SQLITE_PRIVATE Expr *sqlite3LimitWhere(Parse*,SrcList*,Expr*,ExprList*,Expr*,char*); #endif SQLITE_PRIVATE void sqlite3CodeChangeCount(Vdbe*,int,const char*); |
︙ | ︙ | |||
20949 20950 20951 20952 20953 20954 20955 20956 20957 20958 20959 20960 20961 20962 | SQLITE_PRIVATE int sqlite3GetUInt32(const char*, u32*); SQLITE_PRIVATE int sqlite3Atoi(const char*); #ifndef SQLITE_OMIT_UTF16 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar); #endif SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte); SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8**); SQLITE_PRIVATE LogEst sqlite3LogEst(u64); SQLITE_PRIVATE LogEst sqlite3LogEstAdd(LogEst,LogEst); SQLITE_PRIVATE LogEst sqlite3LogEstFromDouble(double); SQLITE_PRIVATE u64 sqlite3LogEstToInt(LogEst); SQLITE_PRIVATE VList *sqlite3VListAdd(sqlite3*,VList*,const char*,int,int); SQLITE_PRIVATE const char *sqlite3VListNumToName(VList*,int); SQLITE_PRIVATE int sqlite3VListNameToNum(VList*,const char*,int); | > | 21081 21082 21083 21084 21085 21086 21087 21088 21089 21090 21091 21092 21093 21094 21095 | SQLITE_PRIVATE int sqlite3GetUInt32(const char*, u32*); SQLITE_PRIVATE int sqlite3Atoi(const char*); #ifndef SQLITE_OMIT_UTF16 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar); #endif SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte); SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8**); SQLITE_PRIVATE int sqlite3Utf8ReadLimited(const u8*, int, u32*); SQLITE_PRIVATE LogEst sqlite3LogEst(u64); SQLITE_PRIVATE LogEst sqlite3LogEstAdd(LogEst,LogEst); SQLITE_PRIVATE LogEst sqlite3LogEstFromDouble(double); SQLITE_PRIVATE u64 sqlite3LogEstToInt(LogEst); SQLITE_PRIVATE VList *sqlite3VListAdd(sqlite3*,VList*,const char*,int,int); SQLITE_PRIVATE const char *sqlite3VListNumToName(VList*,int); SQLITE_PRIVATE int sqlite3VListNameToNum(VList*,const char*,int); |
︙ | ︙ | |||
21295 21296 21297 21298 21299 21300 21301 21302 21303 21304 21305 21306 21307 21308 | SQLITE_PRIVATE int sqlite3WalDefaultHook(void*,sqlite3*,const char*,int); #endif #ifndef SQLITE_OMIT_CTE SQLITE_PRIVATE Cte *sqlite3CteNew(Parse*,Token*,ExprList*,Select*,u8); SQLITE_PRIVATE void sqlite3CteDelete(sqlite3*,Cte*); SQLITE_PRIVATE With *sqlite3WithAdd(Parse*,With*,Cte*); SQLITE_PRIVATE void sqlite3WithDelete(sqlite3*,With*); SQLITE_PRIVATE With *sqlite3WithPush(Parse*, With*, u8); #else # define sqlite3CteNew(P,T,E,S) ((void*)0) # define sqlite3CteDelete(D,C) # define sqlite3CteWithAdd(P,W,C) ((void*)0) # define sqlite3WithDelete(x,y) # define sqlite3WithPush(x,y,z) ((void*)0) | > | 21428 21429 21430 21431 21432 21433 21434 21435 21436 21437 21438 21439 21440 21441 21442 | SQLITE_PRIVATE int sqlite3WalDefaultHook(void*,sqlite3*,const char*,int); #endif #ifndef SQLITE_OMIT_CTE SQLITE_PRIVATE Cte *sqlite3CteNew(Parse*,Token*,ExprList*,Select*,u8); SQLITE_PRIVATE void sqlite3CteDelete(sqlite3*,Cte*); SQLITE_PRIVATE With *sqlite3WithAdd(Parse*,With*,Cte*); SQLITE_PRIVATE void sqlite3WithDelete(sqlite3*,With*); SQLITE_PRIVATE void sqlite3WithDeleteGeneric(sqlite3*,void*); SQLITE_PRIVATE With *sqlite3WithPush(Parse*, With*, u8); #else # define sqlite3CteNew(P,T,E,S) ((void*)0) # define sqlite3CteDelete(D,C) # define sqlite3CteWithAdd(P,W,C) ((void*)0) # define sqlite3WithDelete(x,y) # define sqlite3WithPush(x,y,z) ((void*)0) |
︙ | ︙ | |||
22672 22673 22674 22675 22676 22677 22678 22679 22680 22681 22682 22683 22684 22685 | 1, /* bCoreMutex */ SQLITE_THREADSAFE==1, /* bFullMutex */ SQLITE_USE_URI, /* bOpenUri */ SQLITE_ALLOW_COVERING_INDEX_SCAN, /* bUseCis */ 0, /* bSmallMalloc */ 1, /* bExtraSchemaChecks */ sizeof(LONGDOUBLE_TYPE)>8, /* bUseLongDouble */ 0x7ffffffe, /* mxStrlen */ 0, /* neverCorrupt */ SQLITE_DEFAULT_LOOKASIDE, /* szLookaside, nLookaside */ SQLITE_STMTJRNL_SPILL, /* nStmtSpill */ {0,0,0,0,0,0,0,0}, /* m */ {0,0,0,0,0,0,0,0,0}, /* mutex */ {0,0,0,0,0,0,0,0,0,0,0,0,0},/* pcache2 */ | > > > | 22806 22807 22808 22809 22810 22811 22812 22813 22814 22815 22816 22817 22818 22819 22820 22821 22822 | 1, /* bCoreMutex */ SQLITE_THREADSAFE==1, /* bFullMutex */ SQLITE_USE_URI, /* bOpenUri */ SQLITE_ALLOW_COVERING_INDEX_SCAN, /* bUseCis */ 0, /* bSmallMalloc */ 1, /* bExtraSchemaChecks */ sizeof(LONGDOUBLE_TYPE)>8, /* bUseLongDouble */ #ifdef SQLITE_DEBUG 0, /* bJsonSelfcheck */ #endif 0x7ffffffe, /* mxStrlen */ 0, /* neverCorrupt */ SQLITE_DEFAULT_LOOKASIDE, /* szLookaside, nLookaside */ SQLITE_STMTJRNL_SPILL, /* nStmtSpill */ {0,0,0,0,0,0,0,0}, /* m */ {0,0,0,0,0,0,0,0,0}, /* mutex */ {0,0,0,0,0,0,0,0,0,0,0,0,0},/* pcache2 */ |
︙ | ︙ | |||
23924 23925 23926 23927 23928 23929 23930 | case SQLITE_DBSTATUS_CACHE_SPILL: op = SQLITE_DBSTATUS_CACHE_WRITE+1; /* no break */ deliberate_fall_through case SQLITE_DBSTATUS_CACHE_HIT: case SQLITE_DBSTATUS_CACHE_MISS: case SQLITE_DBSTATUS_CACHE_WRITE:{ int i; | | | | 24061 24062 24063 24064 24065 24066 24067 24068 24069 24070 24071 24072 24073 24074 24075 24076 24077 24078 24079 24080 24081 24082 24083 24084 24085 24086 24087 24088 | case SQLITE_DBSTATUS_CACHE_SPILL: op = SQLITE_DBSTATUS_CACHE_WRITE+1; /* no break */ deliberate_fall_through case SQLITE_DBSTATUS_CACHE_HIT: case SQLITE_DBSTATUS_CACHE_MISS: case SQLITE_DBSTATUS_CACHE_WRITE:{ int i; u64 nRet = 0; assert( SQLITE_DBSTATUS_CACHE_MISS==SQLITE_DBSTATUS_CACHE_HIT+1 ); assert( SQLITE_DBSTATUS_CACHE_WRITE==SQLITE_DBSTATUS_CACHE_HIT+2 ); for(i=0; i<db->nDb; i++){ if( db->aDb[i].pBt ){ Pager *pPager = sqlite3BtreePager(db->aDb[i].pBt); sqlite3PagerCacheStat(pPager, op, resetFlag, &nRet); } } *pHighwater = 0; /* IMP: R-42420-56072 */ /* IMP: R-54100-20147 */ /* IMP: R-29431-39229 */ *pCurrent = (int)nRet & 0x7fffffff; break; } /* Set *pCurrent to non-zero if there are unresolved deferred foreign ** key constraints. Set *pCurrent to zero if all foreign key constraints ** have been satisfied. The *pHighwater is always set to zero. */ |
︙ | ︙ | |||
25006 25007 25008 25009 25010 25011 25012 25013 25014 25015 25016 25017 25018 25019 | for(i=1; i<argc; i++){ z = sqlite3_value_text(argv[i]); n = sqlite3_value_bytes(argv[i]); if( z==0 || parseModifier(context, (char*)z, n, p, i) ) return 1; } computeJD(p); if( p->isError || !validJulianDay(p->iJD) ) return 1; return 0; } /* ** The following routines implement the various date and time functions ** of SQLite. | > > > > > > | 25143 25144 25145 25146 25147 25148 25149 25150 25151 25152 25153 25154 25155 25156 25157 25158 25159 25160 25161 25162 | for(i=1; i<argc; i++){ z = sqlite3_value_text(argv[i]); n = sqlite3_value_bytes(argv[i]); if( z==0 || parseModifier(context, (char*)z, n, p, i) ) return 1; } computeJD(p); if( p->isError || !validJulianDay(p->iJD) ) return 1; if( argc==1 && p->validYMD && p->D>28 ){ /* Make sure a YYYY-MM-DD is normalized. ** Example: 2023-02-31 -> 2023-03-03 */ assert( p->validJD ); p->validYMD = 0; } return 0; } /* ** The following routines implement the various date and time functions ** of SQLite. |
︙ | ︙ | |||
29447 29448 29449 29450 29451 29452 29453 | ** compiled without mutexes (SQLITE_THREADSAFE=0). */ SQLITE_PRIVATE void sqlite3MemoryBarrier(void){ #if defined(SQLITE_MEMORY_BARRIER) SQLITE_MEMORY_BARRIER; #elif defined(__GNUC__) __sync_synchronize(); | | | 29590 29591 29592 29593 29594 29595 29596 29597 29598 29599 29600 29601 29602 29603 29604 | ** compiled without mutexes (SQLITE_THREADSAFE=0). */ SQLITE_PRIVATE void sqlite3MemoryBarrier(void){ #if defined(SQLITE_MEMORY_BARRIER) SQLITE_MEMORY_BARRIER; #elif defined(__GNUC__) __sync_synchronize(); #elif MSVC_VERSION>=1400 _ReadWriteBarrier(); #elif defined(MemoryBarrier) MemoryBarrier(); #endif } /* |
︙ | ︙ | |||
32034 32035 32036 32037 32038 32039 32040 | va_start(ap,zFormat); sqlite3_str_vappendf(p, zFormat, ap); va_end(ap); } /***************************************************************************** | | | 32177 32178 32179 32180 32181 32182 32183 32184 32185 32186 32187 32188 32189 32190 32191 | va_start(ap,zFormat); sqlite3_str_vappendf(p, zFormat, ap); va_end(ap); } /***************************************************************************** ** Reference counted string/blob storage *****************************************************************************/ /* ** Increase the reference count of the string by one. ** ** The input parameter is returned. */ |
︙ | ︙ | |||
32886 32887 32888 32889 32890 32891 32892 | case TK_BETWEEN: { const Expr *pX, *pY, *pZ; pX = pExpr->pLeft; assert( ExprUseXList(pExpr) ); assert( pExpr->x.pList->nExpr==2 ); pY = pExpr->x.pList->a[0].pExpr; pZ = pExpr->x.pList->a[1].pExpr; | | | 33029 33030 33031 33032 33033 33034 33035 33036 33037 33038 33039 33040 33041 33042 33043 | case TK_BETWEEN: { const Expr *pX, *pY, *pZ; pX = pExpr->pLeft; assert( ExprUseXList(pExpr) ); assert( pExpr->x.pList->nExpr==2 ); pY = pExpr->x.pList->a[0].pExpr; pZ = pExpr->x.pList->a[1].pExpr; sqlite3TreeViewLine(pView, "BETWEEN%s", zFlgs); sqlite3TreeViewExpr(pView, pX, 1); sqlite3TreeViewExpr(pView, pY, 1); sqlite3TreeViewExpr(pView, pZ, 0); break; } case TK_TRIGGER: { /* If the opcode is TK_TRIGGER, then the expression is a reference |
︙ | ︙ | |||
34021 34022 34023 34024 34025 34026 34027 | if( c<0x80 || (c&0xFFFFF800)==0xD800 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } } return c; } | > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > | 34164 34165 34166 34167 34168 34169 34170 34171 34172 34173 34174 34175 34176 34177 34178 34179 34180 34181 34182 34183 34184 34185 34186 34187 34188 34189 34190 34191 34192 34193 34194 34195 34196 34197 34198 34199 34200 34201 34202 34203 34204 34205 34206 34207 34208 34209 | if( c<0x80 || (c&0xFFFFF800)==0xD800 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } } return c; } /* ** Read a single UTF8 character out of buffer z[], but reading no ** more than n characters from the buffer. z[] is not zero-terminated. ** ** Return the number of bytes used to construct the character. ** ** Invalid UTF8 might generate a strange result. No effort is made ** to detect invalid UTF8. ** ** At most 4 bytes will be read out of z[]. The return value will always ** be between 1 and 4. */ SQLITE_PRIVATE int sqlite3Utf8ReadLimited( const u8 *z, int n, u32 *piOut ){ u32 c; int i = 1; assert( n>0 ); c = z[0]; if( c>=0xc0 ){ c = sqlite3Utf8Trans1[c-0xc0]; if( n>4 ) n = 4; while( i<n && (z[i] & 0xc0)==0x80 ){ c = (c<<6) + (0x3f & z[i]); i++; } } *piOut = c; return i; } /* ** If the TRANSLATE_TRACE macro is defined, the value of each Mem is ** printed on stderr on the way into and out of sqlite3VdbeMemTranslate(). */ /* #define TRANSLATE_TRACE 1 */ |
︙ | ︙ | |||
34536 34537 34538 34539 34540 34541 34542 | /* ** Load the sqlite3.iSysErrno field if that is an appropriate thing ** to do based on the SQLite error code in rc. */ SQLITE_PRIVATE void sqlite3SystemError(sqlite3 *db, int rc){ if( rc==SQLITE_IOERR_NOMEM ) return; | | | 34710 34711 34712 34713 34714 34715 34716 34717 34718 34719 34720 34721 34722 34723 34724 | /* ** Load the sqlite3.iSysErrno field if that is an appropriate thing ** to do based on the SQLite error code in rc. */ SQLITE_PRIVATE void sqlite3SystemError(sqlite3 *db, int rc){ if( rc==SQLITE_IOERR_NOMEM ) return; #if defined(SQLITE_USE_SEH) && !defined(SQLITE_OMIT_WAL) if( rc==SQLITE_IOERR_IN_PAGE ){ int ii; int iErr; sqlite3BtreeEnterAll(db); for(ii=0; ii<db->nDb; ii++){ if( db->aDb[ii].pBt ){ iErr = sqlite3PagerWalSystemErrno(sqlite3BtreePager(db->aDb[ii].pBt)); |
︙ | ︙ | |||
36790 36791 36792 36793 36794 36795 36796 | /* 174 */ "VCheck" OpHelp(""), /* 175 */ "VInitIn" OpHelp("r[P2]=ValueList(P1,P3)"), /* 176 */ "VColumn" OpHelp("r[P3]=vcolumn(P2)"), /* 177 */ "VRename" OpHelp(""), /* 178 */ "Pagecount" OpHelp(""), /* 179 */ "MaxPgcnt" OpHelp(""), /* 180 */ "ClrSubtype" OpHelp("r[P1].subtype = 0"), | > > | | | | | | | | 36964 36965 36966 36967 36968 36969 36970 36971 36972 36973 36974 36975 36976 36977 36978 36979 36980 36981 36982 36983 36984 36985 36986 | /* 174 */ "VCheck" OpHelp(""), /* 175 */ "VInitIn" OpHelp("r[P2]=ValueList(P1,P3)"), /* 176 */ "VColumn" OpHelp("r[P3]=vcolumn(P2)"), /* 177 */ "VRename" OpHelp(""), /* 178 */ "Pagecount" OpHelp(""), /* 179 */ "MaxPgcnt" OpHelp(""), /* 180 */ "ClrSubtype" OpHelp("r[P1].subtype = 0"), /* 181 */ "GetSubtype" OpHelp("r[P2] = r[P1].subtype"), /* 182 */ "SetSubtype" OpHelp("r[P2].subtype = r[P1]"), /* 183 */ "FilterAdd" OpHelp("filter(P1) += key(P3@P4)"), /* 184 */ "Trace" OpHelp(""), /* 185 */ "CursorHint" OpHelp(""), /* 186 */ "ReleaseReg" OpHelp("release r[P1@P2] mask P3"), /* 187 */ "Noop" OpHelp(""), /* 188 */ "Explain" OpHelp(""), /* 189 */ "Abortable" OpHelp(""), }; return azName[i]; } #endif /************** End of opcodes.c *********************************************/ /************** Begin file os_kv.c *******************************************/ |
︙ | ︙ | |||
40938 40939 40940 40941 40942 40943 40944 | */ static int afpUnlock(sqlite3_file *id, int eFileLock) { int rc = SQLITE_OK; unixFile *pFile = (unixFile*)id; unixInodeInfo *pInode; afpLockingContext *context = (afpLockingContext *) pFile->lockingContext; int skipShared = 0; | < < < < < < | 41114 41115 41116 41117 41118 41119 41120 41121 41122 41123 41124 41125 41126 41127 41128 41129 41130 41131 41132 41133 41134 41135 41136 41137 41138 41139 41140 41141 41142 | */ static int afpUnlock(sqlite3_file *id, int eFileLock) { int rc = SQLITE_OK; unixFile *pFile = (unixFile*)id; unixInodeInfo *pInode; afpLockingContext *context = (afpLockingContext *) pFile->lockingContext; int skipShared = 0; assert( pFile ); OSTRACE(("UNLOCK %d %d was %d(%d,%d) pid=%d (afp)\n", pFile->h, eFileLock, pFile->eFileLock, pFile->pInode->eFileLock, pFile->pInode->nShared, osGetpid(0))); assert( eFileLock<=SHARED_LOCK ); if( pFile->eFileLock<=eFileLock ){ return SQLITE_OK; } pInode = pFile->pInode; sqlite3_mutex_enter(pInode->pLockMutex); assert( pInode->nShared!=0 ); if( pFile->eFileLock>SHARED_LOCK ){ assert( pInode->eFileLock==pFile->eFileLock ); #ifdef SQLITE_DEBUG /* When reducing a lock such that other processes can start ** reading the database file again, make sure that the ** transaction counter was updated if any part of the database ** file changed. If the transaction counter is not updated, ** other connections to the same file might not realize that |
︙ | ︙ | |||
41007 41008 41009 41010 41011 41012 41013 | /* Decrement the shared lock counter. Release the lock using an ** OS call only when all threads in this same process have released ** the lock. */ unsigned long long sharedLockByte = SHARED_FIRST+pInode->sharedByte; pInode->nShared--; if( pInode->nShared==0 ){ | < < < | 41177 41178 41179 41180 41181 41182 41183 41184 41185 41186 41187 41188 41189 41190 | /* Decrement the shared lock counter. Release the lock using an ** OS call only when all threads in this same process have released ** the lock. */ unsigned long long sharedLockByte = SHARED_FIRST+pInode->sharedByte; pInode->nShared--; if( pInode->nShared==0 ){ if( !skipShared ){ rc = afpSetLock(context->dbPath, pFile, sharedLockByte, 1, 0); } if( !rc ){ pInode->eFileLock = NO_LOCK; pFile->eFileLock = NO_LOCK; } |
︙ | ︙ | |||
41851 41852 41853 41854 41855 41856 41857 41858 41859 41860 41861 41862 41863 41864 41865 | case SQLITE_FCNTL_HAS_MOVED: { *(int*)pArg = fileHasMoved(pFile); return SQLITE_OK; } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT case SQLITE_FCNTL_LOCK_TIMEOUT: { int iOld = pFile->iBusyTimeout; pFile->iBusyTimeout = *(int*)pArg; *(int*)pArg = iOld; return SQLITE_OK; } #endif #if SQLITE_MAX_MMAP_SIZE>0 case SQLITE_FCNTL_MMAP_SIZE: { i64 newLimit = *(i64*)pArg; | > > > > > > | 42018 42019 42020 42021 42022 42023 42024 42025 42026 42027 42028 42029 42030 42031 42032 42033 42034 42035 42036 42037 42038 | case SQLITE_FCNTL_HAS_MOVED: { *(int*)pArg = fileHasMoved(pFile); return SQLITE_OK; } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT case SQLITE_FCNTL_LOCK_TIMEOUT: { int iOld = pFile->iBusyTimeout; #if SQLITE_ENABLE_SETLK_TIMEOUT==1 pFile->iBusyTimeout = *(int*)pArg; #elif SQLITE_ENABLE_SETLK_TIMEOUT==2 pFile->iBusyTimeout = !!(*(int*)pArg); #else # error "SQLITE_ENABLE_SETLK_TIMEOUT must be set to 1 or 2" #endif *(int*)pArg = iOld; return SQLITE_OK; } #endif #if SQLITE_MAX_MMAP_SIZE>0 case SQLITE_FCNTL_MMAP_SIZE: { i64 newLimit = *(i64*)pArg; |
︙ | ︙ | |||
42104 42105 42106 42107 42108 42109 42110 42111 42112 42113 42114 42115 42116 42117 42118 42119 42120 42121 42122 42123 42124 42125 | ** ** hShm ** zFilename ** ** Either unixShmNode.pShmMutex must be held or unixShmNode.nRef==0 and ** unixMutexHeld() is true when reading or writing any other field ** in this structure. */ struct unixShmNode { unixInodeInfo *pInode; /* unixInodeInfo that owns this SHM node */ sqlite3_mutex *pShmMutex; /* Mutex to access this object */ char *zFilename; /* Name of the mmapped file */ int hShm; /* Open file descriptor */ int szRegion; /* Size of shared-memory regions */ u16 nRegion; /* Size of array apRegion */ u8 isReadonly; /* True if read-only */ u8 isUnlocked; /* True if no DMS lock held */ char **apRegion; /* Array of mapped shared-memory regions */ int nRef; /* Number of unixShm objects pointing to this */ unixShm *pFirst; /* All unixShm objects pointing to this */ int aLock[SQLITE_SHM_NLOCK]; /* # shared locks on slot, -1==excl lock */ #ifdef SQLITE_DEBUG | > > > > > > > > > > > > > > > > > > > > > > < < | 42277 42278 42279 42280 42281 42282 42283 42284 42285 42286 42287 42288 42289 42290 42291 42292 42293 42294 42295 42296 42297 42298 42299 42300 42301 42302 42303 42304 42305 42306 42307 42308 42309 42310 42311 42312 42313 42314 42315 42316 42317 42318 42319 42320 42321 42322 42323 42324 42325 42326 42327 | ** ** hShm ** zFilename ** ** Either unixShmNode.pShmMutex must be held or unixShmNode.nRef==0 and ** unixMutexHeld() is true when reading or writing any other field ** in this structure. ** ** aLock[SQLITE_SHM_NLOCK]: ** This array records the various locks held by clients on each of the ** SQLITE_SHM_NLOCK slots. If the aLock[] entry is set to 0, then no ** locks are held by the process on this slot. If it is set to -1, then ** some client holds an EXCLUSIVE lock on the locking slot. If the aLock[] ** value is set to a positive value, then it is the number of shared ** locks currently held on the slot. ** ** aMutex[SQLITE_SHM_NLOCK]: ** Normally, when SQLITE_ENABLE_SETLK_TIMEOUT is not defined, mutex ** pShmMutex is used to protect the aLock[] array and the right to ** call fcntl() on unixShmNode.hShm to obtain or release locks. ** ** If SQLITE_ENABLE_SETLK_TIMEOUT is defined though, we use an array ** of mutexes - one for each locking slot. To read or write locking ** slot aLock[iSlot], the caller must hold the corresponding mutex ** aMutex[iSlot]. Similarly, to call fcntl() to obtain or release a ** lock corresponding to slot iSlot, mutex aMutex[iSlot] must be held. */ struct unixShmNode { unixInodeInfo *pInode; /* unixInodeInfo that owns this SHM node */ sqlite3_mutex *pShmMutex; /* Mutex to access this object */ char *zFilename; /* Name of the mmapped file */ int hShm; /* Open file descriptor */ int szRegion; /* Size of shared-memory regions */ u16 nRegion; /* Size of array apRegion */ u8 isReadonly; /* True if read-only */ u8 isUnlocked; /* True if no DMS lock held */ char **apRegion; /* Array of mapped shared-memory regions */ int nRef; /* Number of unixShm objects pointing to this */ unixShm *pFirst; /* All unixShm objects pointing to this */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT sqlite3_mutex *aMutex[SQLITE_SHM_NLOCK]; #endif int aLock[SQLITE_SHM_NLOCK]; /* # shared locks on slot, -1==excl lock */ #ifdef SQLITE_DEBUG u8 nextShmId; /* Next available unixShm.id value */ #endif }; /* ** Structure used internally by this VFS to record the state of an ** open shared memory connection. |
︙ | ︙ | |||
42203 42204 42205 42206 42207 42208 42209 | int ofst, /* First byte of the locking range */ int n /* Number of bytes to lock */ ){ unixShmNode *pShmNode; /* Apply locks to this open shared-memory segment */ struct flock f; /* The posix advisory locking structure */ int rc = SQLITE_OK; /* Result code form fcntl() */ | < > > > > > > > > > | > > > > > > > > | > > > | | < < | < < | < < | < < | | | < < < | 42396 42397 42398 42399 42400 42401 42402 42403 42404 42405 42406 42407 42408 42409 42410 42411 42412 42413 42414 42415 42416 42417 42418 42419 42420 42421 42422 42423 42424 42425 42426 42427 42428 42429 42430 42431 42432 42433 42434 42435 42436 42437 42438 42439 42440 42441 42442 42443 42444 42445 42446 42447 42448 42449 42450 42451 42452 42453 42454 42455 42456 42457 42458 42459 42460 42461 42462 42463 42464 42465 42466 42467 42468 42469 42470 42471 42472 42473 42474 42475 42476 42477 | int ofst, /* First byte of the locking range */ int n /* Number of bytes to lock */ ){ unixShmNode *pShmNode; /* Apply locks to this open shared-memory segment */ struct flock f; /* The posix advisory locking structure */ int rc = SQLITE_OK; /* Result code form fcntl() */ pShmNode = pFile->pInode->pShmNode; /* Assert that the parameters are within expected range and that the ** correct mutex or mutexes are held. */ assert( pShmNode->nRef>=0 ); assert( (ofst==UNIX_SHM_DMS && n==1) || (ofst>=UNIX_SHM_BASE && ofst+n<=(UNIX_SHM_BASE+SQLITE_SHM_NLOCK)) ); if( ofst==UNIX_SHM_DMS ){ assert( pShmNode->nRef>0 || unixMutexHeld() ); assert( pShmNode->nRef==0 || sqlite3_mutex_held(pShmNode->pShmMutex) ); }else{ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT int ii; for(ii=ofst-UNIX_SHM_BASE; ii<ofst-UNIX_SHM_BASE+n; ii++){ assert( sqlite3_mutex_held(pShmNode->aMutex[ii]) ); } #else assert( sqlite3_mutex_held(pShmNode->pShmMutex) ); assert( pShmNode->nRef>0 ); #endif } /* Shared locks never span more than one byte */ assert( n==1 || lockType!=F_RDLCK ); /* Locks are within range */ assert( n>=1 && n<=SQLITE_SHM_NLOCK ); assert( ofst>=UNIX_SHM_BASE && ofst<=(UNIX_SHM_DMS+SQLITE_SHM_NLOCK) ); if( pShmNode->hShm>=0 ){ int res; /* Initialize the locking parameters */ f.l_type = lockType; f.l_whence = SEEK_SET; f.l_start = ofst; f.l_len = n; res = osSetPosixAdvisoryLock(pShmNode->hShm, &f, pFile); if( res==-1 ){ #if defined(SQLITE_ENABLE_SETLK_TIMEOUT) && SQLITE_ENABLE_SETLK_TIMEOUT==1 rc = (pFile->iBusyTimeout ? SQLITE_BUSY_TIMEOUT : SQLITE_BUSY); #else rc = SQLITE_BUSY; #endif } } /* Do debug tracing */ #ifdef SQLITE_DEBUG OSTRACE(("SHM-LOCK ")); if( rc==SQLITE_OK ){ if( lockType==F_UNLCK ){ OSTRACE(("unlock %d..%d ok\n", ofst, ofst+n-1)); }else if( lockType==F_RDLCK ){ OSTRACE(("read-lock %d..%d ok\n", ofst, ofst+n-1)); }else{ assert( lockType==F_WRLCK ); OSTRACE(("write-lock %d..%d ok\n", ofst, ofst+n-1)); } }else{ if( lockType==F_UNLCK ){ OSTRACE(("unlock %d..%d failed\n", ofst, ofst+n-1)); }else if( lockType==F_RDLCK ){ OSTRACE(("read-lock %d..%d failed\n", ofst, ofst+n-1)); }else{ assert( lockType==F_WRLCK ); OSTRACE(("write-lock %d..%d failed\n", ofst, ofst+n-1)); } } #endif return rc; } /* |
︙ | ︙ | |||
42300 42301 42302 42303 42304 42305 42306 42307 42308 42309 42310 42311 42312 42313 | unixShmNode *p = pFd->pInode->pShmNode; assert( unixMutexHeld() ); if( p && ALWAYS(p->nRef==0) ){ int nShmPerMap = unixShmRegionPerMap(); int i; assert( p->pInode==pFd->pInode ); sqlite3_mutex_free(p->pShmMutex); for(i=0; i<p->nRegion; i+=nShmPerMap){ if( p->hShm>=0 ){ osMunmap(p->apRegion[i], p->szRegion); }else{ sqlite3_free(p->apRegion[i]); } } | > > > > > | 42501 42502 42503 42504 42505 42506 42507 42508 42509 42510 42511 42512 42513 42514 42515 42516 42517 42518 42519 | unixShmNode *p = pFd->pInode->pShmNode; assert( unixMutexHeld() ); if( p && ALWAYS(p->nRef==0) ){ int nShmPerMap = unixShmRegionPerMap(); int i; assert( p->pInode==pFd->pInode ); sqlite3_mutex_free(p->pShmMutex); #ifdef SQLITE_ENABLE_SETLK_TIMEOUT for(i=0; i<SQLITE_SHM_NLOCK; i++){ sqlite3_mutex_free(p->aMutex[i]); } #endif for(i=0; i<p->nRegion; i+=nShmPerMap){ if( p->hShm>=0 ){ osMunmap(p->apRegion[i], p->szRegion); }else{ sqlite3_free(p->apRegion[i]); } } |
︙ | ︙ | |||
42359 42360 42361 42362 42363 42364 42365 42366 42367 42368 42369 42370 42371 42372 42373 | if( osFcntl(pShmNode->hShm, F_GETLK, &lock)!=0 ) { rc = SQLITE_IOERR_LOCK; }else if( lock.l_type==F_UNLCK ){ if( pShmNode->isReadonly ){ pShmNode->isUnlocked = 1; rc = SQLITE_READONLY_CANTINIT; }else{ rc = unixShmSystemLock(pDbFd, F_WRLCK, UNIX_SHM_DMS, 1); /* The first connection to attach must truncate the -shm file. We ** truncate to 3 bytes (an arbitrary small number, less than the ** -shm header size) rather than 0 as a system debugging aid, to ** help detect if a -shm file truncation is legitimate or is the work ** or a rogue process. */ if( rc==SQLITE_OK && robust_ftruncate(pShmNode->hShm, 3) ){ rc = unixLogError(SQLITE_IOERR_SHMOPEN,"ftruncate",pShmNode->zFilename); | > > > > > > > > > > > > > | 42565 42566 42567 42568 42569 42570 42571 42572 42573 42574 42575 42576 42577 42578 42579 42580 42581 42582 42583 42584 42585 42586 42587 42588 42589 42590 42591 42592 | if( osFcntl(pShmNode->hShm, F_GETLK, &lock)!=0 ) { rc = SQLITE_IOERR_LOCK; }else if( lock.l_type==F_UNLCK ){ if( pShmNode->isReadonly ){ pShmNode->isUnlocked = 1; rc = SQLITE_READONLY_CANTINIT; }else{ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT /* Do not use a blocking lock here. If the lock cannot be obtained ** immediately, it means some other connection is truncating the ** *-shm file. And after it has done so, it will not release its ** lock, but only downgrade it to a shared lock. So no point in ** blocking here. The call below to obtain the shared DMS lock may ** use a blocking lock. */ int iSaveTimeout = pDbFd->iBusyTimeout; pDbFd->iBusyTimeout = 0; #endif rc = unixShmSystemLock(pDbFd, F_WRLCK, UNIX_SHM_DMS, 1); #ifdef SQLITE_ENABLE_SETLK_TIMEOUT pDbFd->iBusyTimeout = iSaveTimeout; #endif /* The first connection to attach must truncate the -shm file. We ** truncate to 3 bytes (an arbitrary small number, less than the ** -shm header size) rather than 0 as a system debugging aid, to ** help detect if a -shm file truncation is legitimate or is the work ** or a rogue process. */ if( rc==SQLITE_OK && robust_ftruncate(pShmNode->hShm, 3) ){ rc = unixLogError(SQLITE_IOERR_SHMOPEN,"ftruncate",pShmNode->zFilename); |
︙ | ︙ | |||
42480 42481 42482 42483 42484 42485 42486 42487 42488 42489 42490 42491 42492 42493 | pShmNode->pInode = pDbFd->pInode; if( sqlite3GlobalConfig.bCoreMutex ){ pShmNode->pShmMutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST); if( pShmNode->pShmMutex==0 ){ rc = SQLITE_NOMEM_BKPT; goto shm_open_err; } } if( pInode->bProcessLock==0 ){ if( 0==sqlite3_uri_boolean(pDbFd->zPath, "readonly_shm", 0) ){ pShmNode->hShm = robust_open(zShm, O_RDWR|O_CREAT|O_NOFOLLOW, (sStat.st_mode&0777)); } | > > > > > > > > > > > > | 42699 42700 42701 42702 42703 42704 42705 42706 42707 42708 42709 42710 42711 42712 42713 42714 42715 42716 42717 42718 42719 42720 42721 42722 42723 42724 | pShmNode->pInode = pDbFd->pInode; if( sqlite3GlobalConfig.bCoreMutex ){ pShmNode->pShmMutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST); if( pShmNode->pShmMutex==0 ){ rc = SQLITE_NOMEM_BKPT; goto shm_open_err; } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT { int ii; for(ii=0; ii<SQLITE_SHM_NLOCK; ii++){ pShmNode->aMutex[ii] = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST); if( pShmNode->aMutex[ii]==0 ){ rc = SQLITE_NOMEM_BKPT; goto shm_open_err; } } } #endif } if( pInode->bProcessLock==0 ){ if( 0==sqlite3_uri_boolean(pDbFd->zPath, "readonly_shm", 0) ){ pShmNode->hShm = robust_open(zShm, O_RDWR|O_CREAT|O_NOFOLLOW, (sStat.st_mode&0777)); } |
︙ | ︙ | |||
42701 42702 42703 42704 42705 42706 42707 42708 42709 | ** held by each client. Return true if it does, or false otherwise. This ** is to be used in an assert(). e.g. ** ** assert( assertLockingArrayOk(pShmNode) ); */ #ifdef SQLITE_DEBUG static int assertLockingArrayOk(unixShmNode *pShmNode){ unixShm *pX; int aLock[SQLITE_SHM_NLOCK]; | > > > < > | | | 42932 42933 42934 42935 42936 42937 42938 42939 42940 42941 42942 42943 42944 42945 42946 42947 42948 42949 42950 42951 42952 42953 42954 42955 42956 42957 42958 42959 42960 42961 42962 42963 42964 42965 42966 42967 42968 42969 42970 42971 42972 42973 42974 42975 42976 42977 42978 42979 42980 42981 42982 42983 42984 42985 42986 42987 42988 42989 42990 | ** held by each client. Return true if it does, or false otherwise. This ** is to be used in an assert(). e.g. ** ** assert( assertLockingArrayOk(pShmNode) ); */ #ifdef SQLITE_DEBUG static int assertLockingArrayOk(unixShmNode *pShmNode){ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT return 1; #else unixShm *pX; int aLock[SQLITE_SHM_NLOCK]; memset(aLock, 0, sizeof(aLock)); for(pX=pShmNode->pFirst; pX; pX=pX->pNext){ int i; for(i=0; i<SQLITE_SHM_NLOCK; i++){ if( pX->exclMask & (1<<i) ){ assert( aLock[i]==0 ); aLock[i] = -1; }else if( pX->sharedMask & (1<<i) ){ assert( aLock[i]>=0 ); aLock[i]++; } } } assert( 0==memcmp(pShmNode->aLock, aLock, sizeof(aLock)) ); return (memcmp(pShmNode->aLock, aLock, sizeof(aLock))==0); #endif } #endif /* ** Change the lock state for a shared-memory segment. ** ** Note that the relationship between SHARED and EXCLUSIVE locks is a little ** different here than in posix. In xShmLock(), one can go from unlocked ** to shared and back or from unlocked to exclusive and back. But one may ** not go from shared to exclusive or from exclusive to shared. */ static int unixShmLock( sqlite3_file *fd, /* Database file holding the shared memory */ int ofst, /* First lock to acquire or release */ int n, /* Number of locks to acquire or release */ int flags /* What to do with the lock */ ){ unixFile *pDbFd = (unixFile*)fd; /* Connection holding shared memory */ unixShm *p; /* The shared memory being locked */ unixShmNode *pShmNode; /* The underlying file iNode */ int rc = SQLITE_OK; /* Result code */ u16 mask = (1<<(ofst+n)) - (1<<ofst); /* Mask of locks to take or release */ int *aLock; p = pDbFd->pShm; if( p==0 ) return SQLITE_IOERR_SHMLOCK; pShmNode = p->pShmNode; if( NEVER(pShmNode==0) ) return SQLITE_IOERR_SHMLOCK; aLock = pShmNode->aLock; |
︙ | ︙ | |||
42777 42778 42779 42780 42781 42782 42783 | ** In other words, if this is a blocking lock, none of the locks that ** occur later in the above list than the lock being obtained may be ** held. ** ** It is not permitted to block on the RECOVER lock. */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT | > > | | | | | | > > > > > > > > > > > | > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > | > | > | < < > | > > > > > > > > > | | | > > | | | | | | < | < < < < < < | < | | | > | < < | > | | | | | | | | | > > > | > > > | | | < | < | | | | | | | | | | < | | | | | | | | > > > > > > > > > | > > > | 43011 43012 43013 43014 43015 43016 43017 43018 43019 43020 43021 43022 43023 43024 43025 43026 43027 43028 43029 43030 43031 43032 43033 43034 43035 43036 43037 43038 43039 43040 43041 43042 43043 43044 43045 43046 43047 43048 43049 43050 43051 43052 43053 43054 43055 43056 43057 43058 43059 43060 43061 43062 43063 43064 43065 43066 43067 43068 43069 43070 43071 43072 43073 43074 43075 43076 43077 43078 43079 43080 43081 43082 43083 43084 43085 43086 43087 43088 43089 43090 43091 43092 43093 43094 43095 43096 43097 43098 43099 43100 43101 43102 43103 43104 43105 43106 43107 43108 43109 43110 43111 43112 43113 43114 43115 43116 43117 43118 43119 43120 43121 43122 43123 43124 43125 43126 43127 43128 43129 43130 43131 43132 43133 43134 43135 43136 43137 43138 43139 43140 43141 43142 43143 43144 43145 43146 43147 43148 43149 43150 43151 43152 43153 43154 43155 43156 43157 43158 43159 43160 43161 43162 43163 43164 43165 43166 43167 43168 43169 | ** In other words, if this is a blocking lock, none of the locks that ** occur later in the above list than the lock being obtained may be ** held. ** ** It is not permitted to block on the RECOVER lock. */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT { u16 lockMask = (p->exclMask|p->sharedMask); assert( (flags & SQLITE_SHM_UNLOCK) || pDbFd->iBusyTimeout==0 || ( (ofst!=2) /* not RECOVER */ && (ofst!=1 || lockMask==0 || lockMask==2) && (ofst!=0 || lockMask<3) && (ofst<3 || lockMask<(1<<ofst)) )); } #endif /* Check if there is any work to do. There are three cases: ** ** a) An unlock operation where there are locks to unlock, ** b) An shared lock where the requested lock is not already held ** c) An exclusive lock where the requested lock is not already held ** ** The SQLite core never requests an exclusive lock that it already holds. ** This is assert()ed below. */ assert( flags!=(SQLITE_SHM_EXCLUSIVE|SQLITE_SHM_LOCK) || 0==(p->exclMask & mask) ); if( ((flags & SQLITE_SHM_UNLOCK) && ((p->exclMask|p->sharedMask) & mask)) || (flags==(SQLITE_SHM_SHARED|SQLITE_SHM_LOCK) && 0==(p->sharedMask & mask)) || (flags==(SQLITE_SHM_EXCLUSIVE|SQLITE_SHM_LOCK)) ){ /* Take the required mutexes. In SETLK_TIMEOUT mode (blocking locks), if ** this is an attempt on an exclusive lock use sqlite3_mutex_try(). If any ** other thread is holding this mutex, then it is either holding or about ** to hold a lock exclusive to the one being requested, and we may ** therefore return SQLITE_BUSY to the caller. ** ** Doing this prevents some deadlock scenarios. For example, thread 1 may ** be a checkpointer blocked waiting on the WRITER lock. And thread 2 ** may be a normal SQL client upgrading to a write transaction. In this ** case thread 2 does a non-blocking request for the WRITER lock. But - ** if it were to use sqlite3_mutex_enter() then it would effectively ** become a (doomed) blocking request, as thread 2 would block until thread ** 1 obtained WRITER and released the mutex. Since thread 2 already holds ** a lock on a read-locking slot at this point, this breaks the ** anti-deadlock rules (see above). */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT int iMutex; for(iMutex=ofst; iMutex<ofst+n; iMutex++){ if( flags==(SQLITE_SHM_LOCK|SQLITE_SHM_EXCLUSIVE) ){ rc = sqlite3_mutex_try(pShmNode->aMutex[iMutex]); if( rc!=SQLITE_OK ) goto leave_shmnode_mutexes; }else{ sqlite3_mutex_enter(pShmNode->aMutex[iMutex]); } } #else sqlite3_mutex_enter(pShmNode->pShmMutex); #endif if( ALWAYS(rc==SQLITE_OK) ){ if( flags & SQLITE_SHM_UNLOCK ){ /* Case (a) - unlock. */ int bUnlock = 1; assert( (p->exclMask & p->sharedMask)==0 ); assert( !(flags & SQLITE_SHM_EXCLUSIVE) || (p->exclMask & mask)==mask ); assert( !(flags & SQLITE_SHM_SHARED) || (p->sharedMask & mask)==mask ); /* If this is a SHARED lock being unlocked, it is possible that other ** clients within this process are holding the same SHARED lock. In ** this case, set bUnlock to 0 so that the posix lock is not removed ** from the file-descriptor below. */ if( flags & SQLITE_SHM_SHARED ){ assert( n==1 ); assert( aLock[ofst]>=1 ); if( aLock[ofst]>1 ){ bUnlock = 0; aLock[ofst]--; p->sharedMask &= ~mask; } } if( bUnlock ){ rc = unixShmSystemLock(pDbFd, F_UNLCK, ofst+UNIX_SHM_BASE, n); if( rc==SQLITE_OK ){ memset(&aLock[ofst], 0, sizeof(int)*n); p->sharedMask &= ~mask; p->exclMask &= ~mask; } } }else if( flags & SQLITE_SHM_SHARED ){ /* Case (b) - a shared lock. */ if( aLock[ofst]<0 ){ /* An exclusive lock is held by some other connection. BUSY. */ rc = SQLITE_BUSY; }else if( aLock[ofst]==0 ){ rc = unixShmSystemLock(pDbFd, F_RDLCK, ofst+UNIX_SHM_BASE, n); } /* Get the local shared locks */ if( rc==SQLITE_OK ){ p->sharedMask |= mask; aLock[ofst]++; } }else{ /* Case (c) - an exclusive lock. */ int ii; assert( flags==(SQLITE_SHM_LOCK|SQLITE_SHM_EXCLUSIVE) ); assert( (p->sharedMask & mask)==0 ); assert( (p->exclMask & mask)==0 ); /* Make sure no sibling connections hold locks that will block this ** lock. If any do, return SQLITE_BUSY right away. */ for(ii=ofst; ii<ofst+n; ii++){ if( aLock[ii] ){ rc = SQLITE_BUSY; break; } } /* Get the exclusive locks at the system level. Then if successful ** also update the in-memory values. */ if( rc==SQLITE_OK ){ rc = unixShmSystemLock(pDbFd, F_WRLCK, ofst+UNIX_SHM_BASE, n); if( rc==SQLITE_OK ){ p->exclMask |= mask; for(ii=ofst; ii<ofst+n; ii++){ aLock[ii] = -1; } } } } assert( assertLockingArrayOk(pShmNode) ); } /* Drop the mutexes acquired above. */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT leave_shmnode_mutexes: for(iMutex--; iMutex>=ofst; iMutex--){ sqlite3_mutex_leave(pShmNode->aMutex[iMutex]); } #else sqlite3_mutex_leave(pShmNode->pShmMutex); #endif } OSTRACE(("SHM-LOCK shmid-%d, pid-%d got %03x,%03x\n", p->id, osGetpid(0), p->sharedMask, p->exclMask)); return rc; } /* ** Implement a memory barrier or memory fence on shared memory. |
︙ | ︙ | |||
57076 57077 57078 57079 57080 57081 57082 | Pgno lckPgno; /* Page number for the locking page */ i64 pageSize; /* Number of bytes in a page */ i64 journalSizeLimit; /* Size limit for persistent journal files */ char *zFilename; /* Name of the database file */ char *zJournal; /* Name of the journal file */ int (*xBusyHandler)(void*); /* Function to call when busy */ void *pBusyHandlerArg; /* Context argument for xBusyHandler */ | | | 57373 57374 57375 57376 57377 57378 57379 57380 57381 57382 57383 57384 57385 57386 57387 | Pgno lckPgno; /* Page number for the locking page */ i64 pageSize; /* Number of bytes in a page */ i64 journalSizeLimit; /* Size limit for persistent journal files */ char *zFilename; /* Name of the database file */ char *zJournal; /* Name of the journal file */ int (*xBusyHandler)(void*); /* Function to call when busy */ void *pBusyHandlerArg; /* Context argument for xBusyHandler */ u32 aStat[4]; /* Total cache hits, misses, writes, spills */ #ifdef SQLITE_TEST int nRead; /* Database pages read */ #endif void (*xReiniter)(DbPage*); /* Call this routine when reloading pages */ int (*xGet)(Pager*,Pgno,DbPage**,int); /* Routine to fetch a patch */ char *pTmpSpace; /* Pager.pageSize bytes of space for tmp use */ PCache *pPCache; /* Pointer to page cache object */ |
︙ | ︙ | |||
57206 57207 57208 57209 57210 57211 57212 | */ SQLITE_PRIVATE int sqlite3PagerDirectReadOk(Pager *pPager, Pgno pgno){ if( pPager->fd->pMethods==0 ) return 0; if( sqlite3PCacheIsDirty(pPager->pPCache) ) return 0; #ifndef SQLITE_OMIT_WAL if( pPager->pWal ){ u32 iRead = 0; | < | | | 57503 57504 57505 57506 57507 57508 57509 57510 57511 57512 57513 57514 57515 57516 57517 57518 | */ SQLITE_PRIVATE int sqlite3PagerDirectReadOk(Pager *pPager, Pgno pgno){ if( pPager->fd->pMethods==0 ) return 0; if( sqlite3PCacheIsDirty(pPager->pPCache) ) return 0; #ifndef SQLITE_OMIT_WAL if( pPager->pWal ){ u32 iRead = 0; (void)sqlite3WalFindFrame(pPager->pWal, pgno, &iRead); return iRead==0; } #endif return 1; } #endif #ifndef SQLITE_OMIT_WAL |
︙ | ︙ | |||
61450 61451 61452 61453 61454 61455 61456 61457 61458 61459 | /* ** Return the sqlite3_file for the main database given the name ** of the corresponding WAL or Journal name as passed into ** xOpen. */ SQLITE_API sqlite3_file *sqlite3_database_file_object(const char *zName){ Pager *pPager; while( zName[-1]!=0 || zName[-2]!=0 || zName[-3]!=0 || zName[-4]!=0 ){ zName--; } | > > > | | 61746 61747 61748 61749 61750 61751 61752 61753 61754 61755 61756 61757 61758 61759 61760 61761 61762 61763 61764 61765 61766 | /* ** Return the sqlite3_file for the main database given the name ** of the corresponding WAL or Journal name as passed into ** xOpen. */ SQLITE_API sqlite3_file *sqlite3_database_file_object(const char *zName){ Pager *pPager; const char *p; while( zName[-1]!=0 || zName[-2]!=0 || zName[-3]!=0 || zName[-4]!=0 ){ zName--; } p = zName - 4 - sizeof(Pager*); assert( EIGHT_BYTE_ALIGNMENT(p) ); pPager = *(Pager**)p; return pPager->fd; } /* ** This function is called after transitioning from PAGER_UNLOCK to ** PAGER_SHARED state. It tests if there is a hot journal present in |
︙ | ︙ | |||
62976 62977 62978 62979 62980 62981 62982 62983 62984 62985 62986 62987 62988 62989 | pList = sqlite3PcacheDirtyList(pPager->pPCache); #ifdef SQLITE_ENABLE_BATCH_ATOMIC_WRITE if( bBatch ){ rc = sqlite3OsFileControl(fd, SQLITE_FCNTL_BEGIN_ATOMIC_WRITE, 0); if( rc==SQLITE_OK ){ rc = pager_write_pagelist(pPager, pList); if( rc==SQLITE_OK ){ rc = sqlite3OsFileControl(fd, SQLITE_FCNTL_COMMIT_ATOMIC_WRITE, 0); } if( rc!=SQLITE_OK ){ sqlite3OsFileControlHint(fd, SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE, 0); } } | > > > > > > > | 63275 63276 63277 63278 63279 63280 63281 63282 63283 63284 63285 63286 63287 63288 63289 63290 63291 63292 63293 63294 63295 | pList = sqlite3PcacheDirtyList(pPager->pPCache); #ifdef SQLITE_ENABLE_BATCH_ATOMIC_WRITE if( bBatch ){ rc = sqlite3OsFileControl(fd, SQLITE_FCNTL_BEGIN_ATOMIC_WRITE, 0); if( rc==SQLITE_OK ){ rc = pager_write_pagelist(pPager, pList); if( rc==SQLITE_OK && pPager->dbSize>pPager->dbFileSize ){ char *pTmp = pPager->pTmpSpace; int szPage = (int)pPager->pageSize; memset(pTmp, 0, szPage); rc = sqlite3OsWrite(pPager->fd, pTmp, szPage, ((i64)pPager->dbSize*pPager->pageSize)-szPage); } if( rc==SQLITE_OK ){ rc = sqlite3OsFileControl(fd, SQLITE_FCNTL_COMMIT_ATOMIC_WRITE, 0); } if( rc!=SQLITE_OK ){ sqlite3OsFileControlHint(fd, SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE, 0); } } |
︙ | ︙ | |||
63210 63211 63212 63213 63214 63215 63216 | static int a[11]; a[0] = sqlite3PcacheRefCount(pPager->pPCache); a[1] = sqlite3PcachePagecount(pPager->pPCache); a[2] = sqlite3PcacheGetCachesize(pPager->pPCache); a[3] = pPager->eState==PAGER_OPEN ? -1 : (int) pPager->dbSize; a[4] = pPager->eState; a[5] = pPager->errCode; | | | | | | 63516 63517 63518 63519 63520 63521 63522 63523 63524 63525 63526 63527 63528 63529 63530 63531 63532 63533 63534 63535 63536 63537 63538 63539 63540 63541 63542 63543 63544 63545 63546 63547 63548 63549 63550 | static int a[11]; a[0] = sqlite3PcacheRefCount(pPager->pPCache); a[1] = sqlite3PcachePagecount(pPager->pPCache); a[2] = sqlite3PcacheGetCachesize(pPager->pPCache); a[3] = pPager->eState==PAGER_OPEN ? -1 : (int) pPager->dbSize; a[4] = pPager->eState; a[5] = pPager->errCode; a[6] = (int)pPager->aStat[PAGER_STAT_HIT] & 0x7fffffff; a[7] = (int)pPager->aStat[PAGER_STAT_MISS] & 0x7fffffff; a[8] = 0; /* Used to be pPager->nOvfl */ a[9] = pPager->nRead; a[10] = (int)pPager->aStat[PAGER_STAT_WRITE] & 0x7fffffff; return a; } #endif /* ** Parameter eStat must be one of SQLITE_DBSTATUS_CACHE_HIT, _MISS, _WRITE, ** or _WRITE+1. The SQLITE_DBSTATUS_CACHE_WRITE+1 case is a translation ** of SQLITE_DBSTATUS_CACHE_SPILL. The _SPILL case is not contiguous because ** it was added later. ** ** Before returning, *pnVal is incremented by the ** current cache hit or miss count, according to the value of eStat. If the ** reset parameter is non-zero, the cache hit or miss count is zeroed before ** returning. */ SQLITE_PRIVATE void sqlite3PagerCacheStat(Pager *pPager, int eStat, int reset, u64 *pnVal){ assert( eStat==SQLITE_DBSTATUS_CACHE_HIT || eStat==SQLITE_DBSTATUS_CACHE_MISS || eStat==SQLITE_DBSTATUS_CACHE_WRITE || eStat==SQLITE_DBSTATUS_CACHE_WRITE+1 ); |
︙ | ︙ | |||
64170 64171 64172 64173 64174 64175 64176 | */ SQLITE_PRIVATE int sqlite3PagerWalFramesize(Pager *pPager){ assert( pPager->eState>=PAGER_READER ); return sqlite3WalFramesize(pPager->pWal); } #endif | | | 64476 64477 64478 64479 64480 64481 64482 64483 64484 64485 64486 64487 64488 64489 64490 | */ SQLITE_PRIVATE int sqlite3PagerWalFramesize(Pager *pPager){ assert( pPager->eState>=PAGER_READER ); return sqlite3WalFramesize(pPager->pWal); } #endif #if defined(SQLITE_USE_SEH) && !defined(SQLITE_OMIT_WAL) SQLITE_PRIVATE int sqlite3PagerWalSystemErrno(Pager *pPager){ return sqlite3WalSystemErrno(pPager->pWal); } #endif #endif /* SQLITE_OMIT_DISKIO */ |
︙ | ︙ | |||
66186 66187 66188 66189 66190 66191 66192 66193 66194 66195 66196 66197 66198 66199 66200 66201 66202 66203 | p = 0; } *pp = p; return rc; } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT /* ** Attempt to enable blocking locks. Blocking locks are enabled only if (a) ** they are supported by the VFS, and (b) the database handle is configured ** with a busy-timeout. Return 1 if blocking locks are successfully enabled, ** or 0 otherwise. */ static int walEnableBlocking(Wal *pWal){ int res = 0; if( pWal->db ){ int tmout = pWal->db->busyTimeout; if( tmout ){ | > > > > > > > > > > > > > < < < < | | 66492 66493 66494 66495 66496 66497 66498 66499 66500 66501 66502 66503 66504 66505 66506 66507 66508 66509 66510 66511 66512 66513 66514 66515 66516 66517 66518 66519 66520 66521 66522 66523 66524 66525 66526 66527 66528 66529 66530 | p = 0; } *pp = p; return rc; } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT /* ** Attempt to enable blocking locks that block for nMs ms. Return 1 if ** blocking locks are successfully enabled, or 0 otherwise. */ static int walEnableBlockingMs(Wal *pWal, int nMs){ int rc = sqlite3OsFileControl( pWal->pDbFd, SQLITE_FCNTL_LOCK_TIMEOUT, (void*)&nMs ); return (rc==SQLITE_OK); } /* ** Attempt to enable blocking locks. Blocking locks are enabled only if (a) ** they are supported by the VFS, and (b) the database handle is configured ** with a busy-timeout. Return 1 if blocking locks are successfully enabled, ** or 0 otherwise. */ static int walEnableBlocking(Wal *pWal){ int res = 0; if( pWal->db ){ int tmout = pWal->db->busyTimeout; if( tmout ){ res = walEnableBlockingMs(pWal, tmout); } } return res; } /* ** Disable blocking locks. |
︙ | ︙ | |||
66250 66251 66252 66253 66254 66255 66256 | /* ** Set the database handle used to determine if blocking locks are required. */ SQLITE_PRIVATE void sqlite3WalDb(Wal *pWal, sqlite3 *db){ pWal->db = db; } | < < < < < < < < < < | | 66565 66566 66567 66568 66569 66570 66571 66572 66573 66574 66575 66576 66577 66578 66579 66580 66581 66582 | /* ** Set the database handle used to determine if blocking locks are required. */ SQLITE_PRIVATE void sqlite3WalDb(Wal *pWal, sqlite3 *db){ pWal->db = db; } #else # define walEnableBlocking(x) 0 # define walDisableBlocking(x) # define walEnableBlockingMs(pWal, ms) 0 # define sqlite3WalDb(pWal, db) #endif /* ifdef SQLITE_ENABLE_SETLK_TIMEOUT */ /* ** Attempt to obtain the exclusive WAL lock defined by parameters lockIdx and ** n. If the attempt fails and parameter xBusy is not NULL, then it is a |
︙ | ︙ | |||
66864 66865 66866 66867 66868 66869 66870 | if( pWal->bShmUnreliable==0 && (pWal->readOnly & WAL_SHM_RDONLY) ){ if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){ walUnlockShared(pWal, WAL_WRITE_LOCK); rc = SQLITE_READONLY_RECOVERY; } }else{ int bWriteLock = pWal->writeLock; | | > > | > | 67169 67170 67171 67172 67173 67174 67175 67176 67177 67178 67179 67180 67181 67182 67183 67184 67185 67186 67187 67188 67189 67190 67191 67192 67193 67194 | if( pWal->bShmUnreliable==0 && (pWal->readOnly & WAL_SHM_RDONLY) ){ if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){ walUnlockShared(pWal, WAL_WRITE_LOCK); rc = SQLITE_READONLY_RECOVERY; } }else{ int bWriteLock = pWal->writeLock; if( bWriteLock || SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){ pWal->writeLock = 1; if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){ badHdr = walIndexTryHdr(pWal, pChanged); if( badHdr ){ /* If the wal-index header is still malformed even while holding ** a WRITE lock, it can only mean that the header is corrupted and ** needs to be reconstructed. So run recovery to do exactly that. ** Disable blocking locks first. */ walDisableBlocking(pWal); rc = walIndexRecover(pWal); *pChanged = 1; } } if( bWriteLock==0 ){ pWal->writeLock = 0; walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1); |
︙ | ︙ | |||
67082 67083 67084 67085 67086 67087 67088 67089 67090 67091 67092 67093 67094 67095 | pWal->bShmUnreliable = 0; sqlite3WalEndReadTransaction(pWal); *pChanged = 1; } return rc; } /* ** Attempt to start a read transaction. This might fail due to a race or ** other transient condition. When that happens, it returns WAL_RETRY to ** indicate to the caller that it is safe to retry immediately. ** ** On success return SQLITE_OK. On a permanent failure (such an ** I/O error or an SQLITE_BUSY because another process is running | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 67390 67391 67392 67393 67394 67395 67396 67397 67398 67399 67400 67401 67402 67403 67404 67405 67406 67407 67408 67409 67410 67411 67412 67413 67414 67415 67416 67417 67418 67419 67420 67421 67422 67423 67424 67425 67426 67427 67428 67429 67430 67431 67432 67433 67434 | pWal->bShmUnreliable = 0; sqlite3WalEndReadTransaction(pWal); *pChanged = 1; } return rc; } /* ** The final argument passed to walTryBeginRead() is of type (int*). The ** caller should invoke walTryBeginRead as follows: ** ** int cnt = 0; ** do { ** rc = walTryBeginRead(..., &cnt); ** }while( rc==WAL_RETRY ); ** ** The final value of "cnt" is of no use to the caller. It is used by ** the implementation of walTryBeginRead() as follows: ** ** + Each time walTryBeginRead() is called, it is incremented. Once ** it reaches WAL_RETRY_PROTOCOL_LIMIT - indicating that walTryBeginRead() ** has many times been invoked and failed with WAL_RETRY - walTryBeginRead() ** returns SQLITE_PROTOCOL. ** ** + If SQLITE_ENABLE_SETLK_TIMEOUT is defined and walTryBeginRead() failed ** because a blocking lock timed out (SQLITE_BUSY_TIMEOUT from the OS ** layer), the WAL_RETRY_BLOCKED_MASK bit is set in "cnt". In this case ** the next invocation of walTryBeginRead() may omit an expected call to ** sqlite3OsSleep(). There has already been a delay when the previous call ** waited on a lock. */ #define WAL_RETRY_PROTOCOL_LIMIT 100 #ifdef SQLITE_ENABLE_SETLK_TIMEOUT # define WAL_RETRY_BLOCKED_MASK 0x10000000 #else # define WAL_RETRY_BLOCKED_MASK 0 #endif /* ** Attempt to start a read transaction. This might fail due to a race or ** other transient condition. When that happens, it returns WAL_RETRY to ** indicate to the caller that it is safe to retry immediately. ** ** On success return SQLITE_OK. On a permanent failure (such an ** I/O error or an SQLITE_BUSY because another process is running |
︙ | ︙ | |||
67132 67133 67134 67135 67136 67137 67138 | ** This routine uses the nBackfill and aReadMark[] fields of the header ** to select a particular WAL_READ_LOCK() that strives to let the ** checkpoint process do as much work as possible. This routine might ** update values of the aReadMark[] array in the header, but if it does ** so it takes care to hold an exclusive lock on the corresponding ** WAL_READ_LOCK() while changing values. */ | | > > > | 67471 67472 67473 67474 67475 67476 67477 67478 67479 67480 67481 67482 67483 67484 67485 67486 67487 67488 67489 67490 67491 67492 67493 67494 | ** This routine uses the nBackfill and aReadMark[] fields of the header ** to select a particular WAL_READ_LOCK() that strives to let the ** checkpoint process do as much work as possible. This routine might ** update values of the aReadMark[] array in the header, but if it does ** so it takes care to hold an exclusive lock on the corresponding ** WAL_READ_LOCK() while changing values. */ static int walTryBeginRead(Wal *pWal, int *pChanged, int useWal, int *pCnt){ volatile WalCkptInfo *pInfo; /* Checkpoint information in wal-index */ u32 mxReadMark; /* Largest aReadMark[] value */ int mxI; /* Index of largest aReadMark[] value */ int i; /* Loop counter */ int rc = SQLITE_OK; /* Return code */ u32 mxFrame; /* Wal frame to lock to */ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT int nBlockTmout = 0; #endif assert( pWal->readLock<0 ); /* Not currently locked */ /* useWal may only be set for read/write connections */ assert( (pWal->readOnly & WAL_SHM_RDONLY)==0 || useWal==0 ); /* Take steps to avoid spinning forever if there is a protocol error. |
︙ | ︙ | |||
67162 67163 67164 67165 67166 67167 67168 | ** After 5 RETRYs, we begin calling sqlite3OsSleep(). The first few ** calls to sqlite3OsSleep() have a delay of 1 microsecond. Really this ** is more of a scheduler yield than an actual delay. But on the 10th ** an subsequent retries, the delays start becoming longer and longer, ** so that on the 100th (and last) RETRY we delay for 323 milliseconds. ** The total delay time before giving up is less than 10 seconds. */ | > | > | | > > > > > > > > > > > > > > > > > > > > > > > > > | 67504 67505 67506 67507 67508 67509 67510 67511 67512 67513 67514 67515 67516 67517 67518 67519 67520 67521 67522 67523 67524 67525 67526 67527 67528 67529 67530 67531 67532 67533 67534 67535 67536 67537 67538 67539 67540 67541 67542 67543 67544 67545 67546 67547 67548 67549 67550 67551 67552 67553 67554 67555 67556 67557 67558 67559 | ** After 5 RETRYs, we begin calling sqlite3OsSleep(). The first few ** calls to sqlite3OsSleep() have a delay of 1 microsecond. Really this ** is more of a scheduler yield than an actual delay. But on the 10th ** an subsequent retries, the delays start becoming longer and longer, ** so that on the 100th (and last) RETRY we delay for 323 milliseconds. ** The total delay time before giving up is less than 10 seconds. */ (*pCnt)++; if( *pCnt>5 ){ int nDelay = 1; /* Pause time in microseconds */ int cnt = (*pCnt & ~WAL_RETRY_BLOCKED_MASK); if( cnt>WAL_RETRY_PROTOCOL_LIMIT ){ VVA_ONLY( pWal->lockError = 1; ) return SQLITE_PROTOCOL; } if( *pCnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39; #ifdef SQLITE_ENABLE_SETLK_TIMEOUT /* In SQLITE_ENABLE_SETLK_TIMEOUT builds, configure the file-descriptor ** to block for locks for approximately nDelay us. This affects three ** locks: (a) the shared lock taken on the DMS slot in os_unix.c (if ** using os_unix.c), (b) the WRITER lock taken in walIndexReadHdr() if the ** first attempted read fails, and (c) the shared lock taken on the ** read-mark. ** ** If the previous call failed due to an SQLITE_BUSY_TIMEOUT error, ** then sleep for the minimum of 1us. The previous call already provided ** an extra delay while it was blocking on the lock. */ nBlockTmout = (nDelay+998) / 1000; if( !useWal && walEnableBlockingMs(pWal, nBlockTmout) ){ if( *pCnt & WAL_RETRY_BLOCKED_MASK ) nDelay = 1; } #endif sqlite3OsSleep(pWal->pVfs, nDelay); *pCnt &= ~WAL_RETRY_BLOCKED_MASK; } if( !useWal ){ assert( rc==SQLITE_OK ); if( pWal->bShmUnreliable==0 ){ rc = walIndexReadHdr(pWal, pChanged); } #ifdef SQLITE_ENABLE_SETLK_TIMEOUT walDisableBlocking(pWal); if( rc==SQLITE_BUSY_TIMEOUT ){ rc = SQLITE_BUSY; *pCnt |= WAL_RETRY_BLOCKED_MASK; } #endif if( rc==SQLITE_BUSY ){ /* If there is not a recovery running in another thread or process ** then convert BUSY errors to WAL_RETRY. If recovery is known to ** be running, convert BUSY to BUSY_RECOVERY. There is a race here ** which might cause WAL_RETRY to be returned even if BUSY_RECOVERY ** would be technically correct. But the race is benign since with ** WAL_RETRY this routine will be called again and will probably be |
︙ | ︙ | |||
67291 67292 67293 67294 67295 67296 67297 67298 67299 | } } if( mxI==0 ){ assert( rc==SQLITE_BUSY || (pWal->readOnly & WAL_SHM_RDONLY)!=0 ); return rc==SQLITE_BUSY ? WAL_RETRY : SQLITE_READONLY_CANTINIT; } rc = walLockShared(pWal, WAL_READ_LOCK(mxI)); if( rc ){ | > > > > > > > > > > | | 67660 67661 67662 67663 67664 67665 67666 67667 67668 67669 67670 67671 67672 67673 67674 67675 67676 67677 67678 67679 67680 67681 67682 67683 67684 67685 67686 | } } if( mxI==0 ){ assert( rc==SQLITE_BUSY || (pWal->readOnly & WAL_SHM_RDONLY)!=0 ); return rc==SQLITE_BUSY ? WAL_RETRY : SQLITE_READONLY_CANTINIT; } (void)walEnableBlockingMs(pWal, nBlockTmout); rc = walLockShared(pWal, WAL_READ_LOCK(mxI)); walDisableBlocking(pWal); if( rc ){ #ifdef SQLITE_ENABLE_SETLK_TIMEOUT if( rc==SQLITE_BUSY_TIMEOUT ){ *pCnt |= WAL_RETRY_BLOCKED_MASK; } #else assert( rc!=SQLITE_BUSY_TIMEOUT ); #endif assert( (rc&0xFF)!=SQLITE_BUSY||rc==SQLITE_BUSY||rc==SQLITE_BUSY_TIMEOUT ); return (rc&0xFF)==SQLITE_BUSY ? WAL_RETRY : rc; } /* Now that the read-lock has been obtained, check that neither the ** value in the aReadMark[] array or the contents of the wal-index ** header have changed. ** ** It is necessary to check that the wal-index header did not change ** between the time it was read and when the shared-lock was obtained |
︙ | ︙ | |||
67481 67482 67483 67484 67485 67486 67487 | return rc; } ckptLock = 1; } #endif do{ | | | 67860 67861 67862 67863 67864 67865 67866 67867 67868 67869 67870 67871 67872 67873 67874 | return rc; } ckptLock = 1; } #endif do{ rc = walTryBeginRead(pWal, pChanged, 0, &cnt); }while( rc==WAL_RETRY ); testcase( (rc&0xff)==SQLITE_BUSY ); testcase( (rc&0xff)==SQLITE_IOERR ); testcase( rc==SQLITE_PROTOCOL ); testcase( rc==SQLITE_OK ); #ifdef SQLITE_ENABLE_SNAPSHOT |
︙ | ︙ | |||
67662 67663 67664 67665 67666 67667 67668 67669 67670 67671 67672 67673 67674 67675 | while( (iH = AtomicLoad(&sLoc.aHash[iKey]))!=0 ){ u32 iFrame = iH + sLoc.iZero; if( iFrame<=iLast && iFrame>=pWal->minFrame && sLoc.aPgno[iH-1]==pgno ){ assert( iFrame>iRead || CORRUPT_DB ); iRead = iFrame; } if( (nCollide--)==0 ){ return SQLITE_CORRUPT_BKPT; } iKey = walNextHash(iKey); } if( iRead ) break; } | > | 68041 68042 68043 68044 68045 68046 68047 68048 68049 68050 68051 68052 68053 68054 68055 | while( (iH = AtomicLoad(&sLoc.aHash[iKey]))!=0 ){ u32 iFrame = iH + sLoc.iZero; if( iFrame<=iLast && iFrame>=pWal->minFrame && sLoc.aPgno[iH-1]==pgno ){ assert( iFrame>iRead || CORRUPT_DB ); iRead = iFrame; } if( (nCollide--)==0 ){ *piRead = 0; return SQLITE_CORRUPT_BKPT; } iKey = walNextHash(iKey); } if( iRead ) break; } |
︙ | ︙ | |||
67965 67966 67967 67968 67969 67970 67971 | } } walUnlockShared(pWal, WAL_READ_LOCK(0)); pWal->readLock = -1; cnt = 0; do{ int notUsed; | | | 68345 68346 68347 68348 68349 68350 68351 68352 68353 68354 68355 68356 68357 68358 68359 | } } walUnlockShared(pWal, WAL_READ_LOCK(0)); pWal->readLock = -1; cnt = 0; do{ int notUsed; rc = walTryBeginRead(pWal, ¬Used, 1, &cnt); }while( rc==WAL_RETRY ); assert( (rc&0xff)!=SQLITE_BUSY ); /* BUSY not possible when useWal==1 */ testcase( (rc&0xff)==SQLITE_IOERR ); testcase( rc==SQLITE_PROTOCOL ); testcase( rc==SQLITE_OK ); } return rc; |
︙ | ︙ | |||
68386 68387 68388 68389 68390 68391 68392 | /* EVIDENCE-OF: R-62920-47450 The busy-handler callback is never invoked ** in the SQLITE_CHECKPOINT_PASSIVE mode. */ assert( eMode!=SQLITE_CHECKPOINT_PASSIVE || xBusy==0 ); if( pWal->readOnly ) return SQLITE_READONLY; WALTRACE(("WAL%p: checkpoint begins\n", pWal)); | | < | | 68766 68767 68768 68769 68770 68771 68772 68773 68774 68775 68776 68777 68778 68779 68780 68781 68782 | /* EVIDENCE-OF: R-62920-47450 The busy-handler callback is never invoked ** in the SQLITE_CHECKPOINT_PASSIVE mode. */ assert( eMode!=SQLITE_CHECKPOINT_PASSIVE || xBusy==0 ); if( pWal->readOnly ) return SQLITE_READONLY; WALTRACE(("WAL%p: checkpoint begins\n", pWal)); /* Enable blocking locks, if possible. */ sqlite3WalDb(pWal, db); if( xBusy2 ) (void)walEnableBlocking(pWal); /* IMPLEMENTATION-OF: R-62028-47212 All calls obtain an exclusive ** "checkpoint" lock on the database file. ** EVIDENCE-OF: R-10421-19736 If any other process is running a ** checkpoint operation at the same time, the lock cannot be obtained and ** SQLITE_BUSY is returned. ** EVIDENCE-OF: R-53820-33897 Even if there is a busy-handler configured, |
︙ | ︙ | |||
68430 68431 68432 68433 68434 68435 68436 68437 68438 | } } /* Read the wal-index header. */ SEH_TRY { if( rc==SQLITE_OK ){ walDisableBlocking(pWal); rc = walIndexReadHdr(pWal, &isChanged); | > > > > > | | 68809 68810 68811 68812 68813 68814 68815 68816 68817 68818 68819 68820 68821 68822 68823 68824 68825 68826 68827 68828 68829 68830 | } } /* Read the wal-index header. */ SEH_TRY { if( rc==SQLITE_OK ){ /* For a passive checkpoint, do not re-enable blocking locks after ** reading the wal-index header. A passive checkpoint should not block ** or invoke the busy handler. The only lock such a checkpoint may ** attempt to obtain is a lock on a read-slot, and it should give up ** immediately and do a partial checkpoint if it cannot obtain it. */ walDisableBlocking(pWal); rc = walIndexReadHdr(pWal, &isChanged); if( eMode2!=SQLITE_CHECKPOINT_PASSIVE ) (void)walEnableBlocking(pWal); if( isChanged && pWal->pDbFd->pMethods->iVersion>=3 ){ sqlite3OsUnfetch(pWal->pDbFd, 0, 0); } } /* Copy data from the log to the database file. */ if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
68769 68770 68771 68772 68773 68774 68775 | ** 18 1 File format write version ** 19 1 File format read version ** 20 1 Bytes of unused space at the end of each page ** 21 1 Max embedded payload fraction (must be 64) ** 22 1 Min embedded payload fraction (must be 32) ** 23 1 Min leaf payload fraction (must be 32) ** 24 4 File change counter | | | 69153 69154 69155 69156 69157 69158 69159 69160 69161 69162 69163 69164 69165 69166 69167 | ** 18 1 File format write version ** 19 1 File format read version ** 20 1 Bytes of unused space at the end of each page ** 21 1 Max embedded payload fraction (must be 64) ** 22 1 Min embedded payload fraction (must be 32) ** 23 1 Min leaf payload fraction (must be 32) ** 24 4 File change counter ** 28 4 The size of the database in pages ** 32 4 First freelist page ** 36 4 Number of freelist pages in the file ** 40 60 15 4-byte meta values passed to higher layers ** ** 40 4 Schema cookie ** 44 4 File format of schema layer ** 48 4 Size of page cache |
︙ | ︙ | |||
74896 74897 74898 74899 74900 74901 74902 | ){ sqlite3_file *fd = sqlite3PagerFile(pBt->pPager); u8 aSave[4]; u8 *aWrite = &pBuf[-4]; assert( aWrite>=pBufStart ); /* due to (6) */ memcpy(aSave, aWrite, 4); rc = sqlite3OsRead(fd, aWrite, a+4, (i64)pBt->pageSize*(nextPage-1)); | < | 75280 75281 75282 75283 75284 75285 75286 75287 75288 75289 75290 75291 75292 75293 | ){ sqlite3_file *fd = sqlite3PagerFile(pBt->pPager); u8 aSave[4]; u8 *aWrite = &pBuf[-4]; assert( aWrite>=pBufStart ); /* due to (6) */ memcpy(aSave, aWrite, 4); rc = sqlite3OsRead(fd, aWrite, a+4, (i64)pBt->pageSize*(nextPage-1)); nextPage = get4byte(aWrite); memcpy(aWrite, aSave, 4); }else #endif { DbPage *pDbPage; |
︙ | ︙ | |||
80219 80220 80221 80222 80223 80224 80225 | ** Return 1 if there are 2 or more references to the page and 0 if ** if this is the first reference to the page. ** ** Also check that the page number is in bounds. */ static int checkRef(IntegrityCk *pCheck, Pgno iPage){ if( iPage>pCheck->nCkPage || iPage==0 ){ | < | 80602 80603 80604 80605 80606 80607 80608 80609 80610 80611 80612 80613 80614 80615 | ** Return 1 if there are 2 or more references to the page and 0 if ** if this is the first reference to the page. ** ** Also check that the page number is in bounds. */ static int checkRef(IntegrityCk *pCheck, Pgno iPage){ if( iPage>pCheck->nCkPage || iPage==0 ){ checkAppendMsg(pCheck, "invalid page number %u", iPage); return 1; } if( getPageReferenced(pCheck, iPage) ){ checkAppendMsg(pCheck, "2nd reference to page %u", iPage); return 1; } |
︙ | ︙ | |||
80725 80726 80727 80728 80729 80730 80731 | sCheck.mxErr = mxErr; sqlite3StrAccumInit(&sCheck.errMsg, 0, zErr, sizeof(zErr), SQLITE_MAX_LENGTH); sCheck.errMsg.printfFlags = SQLITE_PRINTF_INTERNAL; if( sCheck.nCkPage==0 ){ goto integrity_ck_cleanup; } | < < < < | | | | < | 81107 81108 81109 81110 81111 81112 81113 81114 81115 81116 81117 81118 81119 81120 81121 81122 81123 81124 | sCheck.mxErr = mxErr; sqlite3StrAccumInit(&sCheck.errMsg, 0, zErr, sizeof(zErr), SQLITE_MAX_LENGTH); sCheck.errMsg.printfFlags = SQLITE_PRINTF_INTERNAL; if( sCheck.nCkPage==0 ){ goto integrity_ck_cleanup; } sCheck.aPgRef = sqlite3MallocZero((sCheck.nCkPage / 8)+ 1); if( !sCheck.aPgRef ){ checkOom(&sCheck); goto integrity_ck_cleanup; } sCheck.heap = (u32*)sqlite3PageMalloc( pBt->pageSize ); if( sCheck.heap==0 ){ checkOom(&sCheck); goto integrity_ck_cleanup; } |
︙ | ︙ | |||
83413 83414 83415 83416 83417 83418 83419 | assert( !ExprHasProperty(p, EP_IntValue) ); pFunc = sqlite3FindFunction(db, p->u.zToken, nVal, enc, 0); #ifdef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION if( pFunc==0 ) return SQLITE_OK; #endif assert( pFunc ); if( (pFunc->funcFlags & (SQLITE_FUNC_CONSTANT|SQLITE_FUNC_SLOCHNG))==0 | | | 83790 83791 83792 83793 83794 83795 83796 83797 83798 83799 83800 83801 83802 83803 83804 | assert( !ExprHasProperty(p, EP_IntValue) ); pFunc = sqlite3FindFunction(db, p->u.zToken, nVal, enc, 0); #ifdef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION if( pFunc==0 ) return SQLITE_OK; #endif assert( pFunc ); if( (pFunc->funcFlags & (SQLITE_FUNC_CONSTANT|SQLITE_FUNC_SLOCHNG))==0 || (pFunc->funcFlags & (SQLITE_FUNC_NEEDCOLL|SQLITE_FUNC_RUNONLY))!=0 ){ return SQLITE_OK; } if( pList ){ apVal = (sqlite3_value**)sqlite3DbMallocZero(db, sizeof(apVal[0]) * nVal); if( apVal==0 ){ |
︙ | ︙ | |||
84137 84138 84139 84140 84141 84142 84143 | ** Other useful labels for breakpoints include: ** test_trace_breakpoint(pc,pOp) ** sqlite3CorruptError(lineno) ** sqlite3MisuseError(lineno) ** sqlite3CantopenError(lineno) */ static void test_addop_breakpoint(int pc, Op *pOp){ | | > | 84514 84515 84516 84517 84518 84519 84520 84521 84522 84523 84524 84525 84526 84527 84528 84529 84530 84531 84532 | ** Other useful labels for breakpoints include: ** test_trace_breakpoint(pc,pOp) ** sqlite3CorruptError(lineno) ** sqlite3MisuseError(lineno) ** sqlite3CantopenError(lineno) */ static void test_addop_breakpoint(int pc, Op *pOp){ static u64 n = 0; (void)pc; (void)pOp; n++; if( n==LARGEST_UINT64 ) abort(); /* so that n is used, preventing a warning */ } #endif /* ** Slow paths for sqlite3VdbeAddOp3() and sqlite3VdbeAddOp4Int() for the ** unusual case when we need to increase the size of the Vdbe.aOp[] array ** before adding the new opcode. |
︙ | ︙ | |||
85325 85326 85327 85328 85329 85330 85331 85332 85333 85334 85335 85336 85337 85338 | } break; } case P4_VTAB : { if( db->pnBytesFreed==0 ) sqlite3VtabUnlock((VTable *)p4); break; } } } /* ** Free the space allocated for aOp and any p4 values allocated for the ** opcodes contained within. If aOp is not NULL it is assumed to contain ** nOp entries. | > > > > | 85703 85704 85705 85706 85707 85708 85709 85710 85711 85712 85713 85714 85715 85716 85717 85718 85719 85720 | } break; } case P4_VTAB : { if( db->pnBytesFreed==0 ) sqlite3VtabUnlock((VTable *)p4); break; } case P4_TABLEREF: { if( db->pnBytesFreed==0 ) sqlite3DeleteTable(db, (Table*)p4); break; } } } /* ** Free the space allocated for aOp and any p4 values allocated for the ** opcodes contained within. If aOp is not NULL it is assumed to contain ** nOp entries. |
︙ | ︙ | |||
85452 85453 85454 85455 85456 85457 85458 | static void SQLITE_NOINLINE vdbeChangeP4Full( Vdbe *p, Op *pOp, const char *zP4, int n ){ if( pOp->p4type ){ | | | 85834 85835 85836 85837 85838 85839 85840 85841 85842 85843 85844 85845 85846 85847 85848 | static void SQLITE_NOINLINE vdbeChangeP4Full( Vdbe *p, Op *pOp, const char *zP4, int n ){ if( pOp->p4type ){ assert( pOp->p4type > P4_FREE_IF_LE ); pOp->p4type = 0; pOp->p4.p = 0; } if( n<0 ){ sqlite3VdbeChangeP4(p, (int)(pOp - p->aOp), zP4, n); }else{ if( n==0 ) n = sqlite3Strlen30(zP4); |
︙ | ︙ | |||
88407 88408 88409 88410 88411 88412 88413 88414 88415 88416 88417 88418 88419 88420 | /* ** Do a comparison between a 64-bit signed integer and a 64-bit floating-point ** number. Return negative, zero, or positive if the first (i64) is less than, ** equal to, or greater than the second (double). */ SQLITE_PRIVATE int sqlite3IntFloatCompare(i64 i, double r){ if( sqlite3Config.bUseLongDouble ){ LONGDOUBLE_TYPE x = (LONGDOUBLE_TYPE)i; testcase( x<r ); testcase( x>r ); testcase( x==r ); return (x<r) ? -1 : (x>r); }else{ | > > > > > | 88789 88790 88791 88792 88793 88794 88795 88796 88797 88798 88799 88800 88801 88802 88803 88804 88805 88806 88807 | /* ** Do a comparison between a 64-bit signed integer and a 64-bit floating-point ** number. Return negative, zero, or positive if the first (i64) is less than, ** equal to, or greater than the second (double). */ SQLITE_PRIVATE int sqlite3IntFloatCompare(i64 i, double r){ if( sqlite3IsNaN(r) ){ /* SQLite considers NaN to be a NULL. And all integer values are greater ** than NULL */ return 1; } if( sqlite3Config.bUseLongDouble ){ LONGDOUBLE_TYPE x = (LONGDOUBLE_TYPE)i; testcase( x<r ); testcase( x>r ); testcase( x==r ); return (x<r) ? -1 : (x>r); }else{ |
︙ | ︙ | |||
89570 89571 89572 89573 89574 89575 89576 | ** Set all the parameters in the compiled SQL statement to NULL. */ SQLITE_API int sqlite3_clear_bindings(sqlite3_stmt *pStmt){ int i; int rc = SQLITE_OK; Vdbe *p = (Vdbe*)pStmt; #if SQLITE_THREADSAFE | | > > > > > > > > | 89957 89958 89959 89960 89961 89962 89963 89964 89965 89966 89967 89968 89969 89970 89971 89972 89973 89974 89975 89976 89977 89978 89979 | ** Set all the parameters in the compiled SQL statement to NULL. */ SQLITE_API int sqlite3_clear_bindings(sqlite3_stmt *pStmt){ int i; int rc = SQLITE_OK; Vdbe *p = (Vdbe*)pStmt; #if SQLITE_THREADSAFE sqlite3_mutex *mutex; #endif #ifdef SQLITE_ENABLE_API_ARMOR if( pStmt==0 ){ return SQLITE_MISUSE_BKPT; } #endif #if SQLITE_THREADSAFE mutex = p->db->mutex; #endif sqlite3_mutex_enter(mutex); for(i=0; i<p->nVar; i++){ sqlite3VdbeMemRelease(&p->aVar[i]); p->aVar[i].flags = MEM_Null; } assert( (p->prepFlags & SQLITE_PREPARE_SAVESQL)!=0 || p->expmask==0 ); |
︙ | ︙ | |||
89949 89950 89951 89952 89953 89954 89955 89956 89957 89958 89959 89960 89961 89962 | sqlite3VdbeMemSetPointer(pOut, pPtr, zPType, xDestructor); } SQLITE_API void sqlite3_result_subtype(sqlite3_context *pCtx, unsigned int eSubtype){ Mem *pOut; #ifdef SQLITE_ENABLE_API_ARMOR if( pCtx==0 ) return; #endif pOut = pCtx->pOut; assert( sqlite3_mutex_held(pOut->db->mutex) ); pOut->eSubtype = eSubtype & 0xff; pOut->flags |= MEM_Subtype; } SQLITE_API void sqlite3_result_text( sqlite3_context *pCtx, | > > > > > > > > > > > > | 90344 90345 90346 90347 90348 90349 90350 90351 90352 90353 90354 90355 90356 90357 90358 90359 90360 90361 90362 90363 90364 90365 90366 90367 90368 90369 | sqlite3VdbeMemSetPointer(pOut, pPtr, zPType, xDestructor); } SQLITE_API void sqlite3_result_subtype(sqlite3_context *pCtx, unsigned int eSubtype){ Mem *pOut; #ifdef SQLITE_ENABLE_API_ARMOR if( pCtx==0 ) return; #endif #if defined(SQLITE_STRICT_SUBTYPE) && SQLITE_STRICT_SUBTYPE+0!=0 if( pCtx->pFunc!=0 && (pCtx->pFunc->funcFlags & SQLITE_RESULT_SUBTYPE)==0 ){ char zErr[200]; sqlite3_snprintf(sizeof(zErr), zErr, "misuse of sqlite3_result_subtype() by %s()", pCtx->pFunc->zName); sqlite3_result_error(pCtx, zErr, -1); return; } #endif /* SQLITE_STRICT_SUBTYPE */ pOut = pCtx->pOut; assert( sqlite3_mutex_held(pOut->db->mutex) ); pOut->eSubtype = eSubtype & 0xff; pOut->flags |= MEM_Subtype; } SQLITE_API void sqlite3_result_text( sqlite3_context *pCtx, |
︙ | ︙ | |||
90348 90349 90350 90351 90352 90353 90354 | /* ** Extract the user data from a sqlite3_context structure and return a ** pointer to it. */ SQLITE_API void *sqlite3_user_data(sqlite3_context *p){ #ifdef SQLITE_ENABLE_API_ARMOR if( p==0 ) return 0; | | < | 90755 90756 90757 90758 90759 90760 90761 90762 90763 90764 90765 90766 90767 90768 90769 90770 | /* ** Extract the user data from a sqlite3_context structure and return a ** pointer to it. */ SQLITE_API void *sqlite3_user_data(sqlite3_context *p){ #ifdef SQLITE_ENABLE_API_ARMOR if( p==0 ) return 0; #endif assert( p && p->pFunc ); return p->pFunc->pUserData; } /* ** Extract the user data from a sqlite3_context structure and return a ** pointer to it. ** |
︙ | ︙ | |||
92267 92268 92269 92270 92271 92272 92273 | ** Other useful labels for breakpoints include: ** test_addop_breakpoint(pc,pOp) ** sqlite3CorruptError(lineno) ** sqlite3MisuseError(lineno) ** sqlite3CantopenError(lineno) */ static void test_trace_breakpoint(int pc, Op *pOp, Vdbe *v){ | | > | 92673 92674 92675 92676 92677 92678 92679 92680 92681 92682 92683 92684 92685 92686 92687 92688 92689 92690 92691 92692 | ** Other useful labels for breakpoints include: ** test_addop_breakpoint(pc,pOp) ** sqlite3CorruptError(lineno) ** sqlite3MisuseError(lineno) ** sqlite3CantopenError(lineno) */ static void test_trace_breakpoint(int pc, Op *pOp, Vdbe *v){ static u64 n = 0; (void)pc; (void)pOp; (void)v; n++; if( n==LARGEST_UINT64 ) abort(); /* So that n is used, preventing a warning */ } #endif /* ** Invoke the VDBE coverage callback, if that callback is defined. This ** feature is used for test suite validation only and does not appear an ** production builds. |
︙ | ︙ | |||
94168 94169 94170 94171 94172 94173 94174 | ** ** To force any register to be an integer, just add 0. */ case OP_AddImm: { /* in1 */ pIn1 = &aMem[pOp->p1]; memAboutToChange(p, pIn1); sqlite3VdbeMemIntegerify(pIn1); | | | 94575 94576 94577 94578 94579 94580 94581 94582 94583 94584 94585 94586 94587 94588 94589 | ** ** To force any register to be an integer, just add 0. */ case OP_AddImm: { /* in1 */ pIn1 = &aMem[pOp->p1]; memAboutToChange(p, pIn1); sqlite3VdbeMemIntegerify(pIn1); *(u64*)&pIn1->u.i += (u64)pOp->p2; break; } /* Opcode: MustBeInt P1 P2 * * * ** ** Force the value in register P1 to be an integer. If the value ** in P1 is not an integer and cannot be converted into an integer |
︙ | ︙ | |||
99046 99047 99048 99049 99050 99051 99052 99053 99054 99055 99056 99057 99058 99059 99060 99061 99062 99063 99064 99065 99066 99067 99068 99069 99070 99071 99072 99073 99074 | ** ** Run the SQL statement or statements specified in the P4 string. ** Disable Auth and Trace callbacks while those statements are running if ** P1 is true. */ case OP_SqlExec: { char *zErr; sqlite3_xauth xAuth; u8 mTrace; sqlite3VdbeIncrWriteCounter(p, 0); db->nSqlExec++; zErr = 0; xAuth = db->xAuth; mTrace = db->mTrace; if( pOp->p1 ){ db->xAuth = 0; db->mTrace = 0; } rc = sqlite3_exec(db, pOp->p4.z, 0, 0, &zErr); db->nSqlExec--; db->xAuth = xAuth; db->mTrace = mTrace; if( zErr || rc ){ sqlite3VdbeError(p, "%s", zErr); sqlite3_free(zErr); if( rc==SQLITE_NOMEM ) goto no_mem; goto abort_due_to_error; } | > > > > > > > > | 99453 99454 99455 99456 99457 99458 99459 99460 99461 99462 99463 99464 99465 99466 99467 99468 99469 99470 99471 99472 99473 99474 99475 99476 99477 99478 99479 99480 99481 99482 99483 99484 99485 99486 99487 99488 99489 | ** ** Run the SQL statement or statements specified in the P4 string. ** Disable Auth and Trace callbacks while those statements are running if ** P1 is true. */ case OP_SqlExec: { char *zErr; #ifndef SQLITE_OMIT_AUTHORIZATION sqlite3_xauth xAuth; #endif u8 mTrace; sqlite3VdbeIncrWriteCounter(p, 0); db->nSqlExec++; zErr = 0; #ifndef SQLITE_OMIT_AUTHORIZATION xAuth = db->xAuth; #endif mTrace = db->mTrace; if( pOp->p1 ){ #ifndef SQLITE_OMIT_AUTHORIZATION db->xAuth = 0; #endif db->mTrace = 0; } rc = sqlite3_exec(db, pOp->p4.z, 0, 0, &zErr); db->nSqlExec--; #ifndef SQLITE_OMIT_AUTHORIZATION db->xAuth = xAuth; #endif db->mTrace = mTrace; if( zErr || rc ){ sqlite3VdbeError(p, "%s", zErr); sqlite3_free(zErr); if( rc==SQLITE_NOMEM ) goto no_mem; goto abort_due_to_error; } |
︙ | ︙ | |||
100289 100290 100291 100292 100293 100294 100295 | goto no_mem; } break; } #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifndef SQLITE_OMIT_VIRTUALTABLE | | | | | > | | | > | < > | > < | 100704 100705 100706 100707 100708 100709 100710 100711 100712 100713 100714 100715 100716 100717 100718 100719 100720 100721 100722 100723 100724 100725 100726 100727 100728 100729 100730 100731 100732 100733 100734 100735 100736 100737 100738 100739 100740 100741 100742 100743 100744 100745 100746 100747 100748 100749 100750 100751 | goto no_mem; } break; } #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifndef SQLITE_OMIT_VIRTUALTABLE /* Opcode: VCheck P1 P2 P3 P4 * ** ** P4 is a pointer to a Table object that is a virtual table in schema P1 ** that supports the xIntegrity() method. This opcode runs the xIntegrity() ** method for that virtual table, using P3 as the integer argument. If ** an error is reported back, the table name is prepended to the error ** message and that message is stored in P2. If no errors are seen, ** register P2 is set to NULL. */ case OP_VCheck: { /* out2 */ Table *pTab; sqlite3_vtab *pVtab; const sqlite3_module *pModule; char *zErr = 0; pOut = &aMem[pOp->p2]; sqlite3VdbeMemSetNull(pOut); /* Innocent until proven guilty */ assert( pOp->p4type==P4_TABLEREF ); pTab = pOp->p4.pTab; assert( pTab!=0 ); assert( pTab->nTabRef>0 ); assert( IsVirtual(pTab) ); if( pTab->u.vtab.p==0 ) break; pVtab = pTab->u.vtab.p->pVtab; assert( pVtab!=0 ); pModule = pVtab->pModule; assert( pModule!=0 ); assert( pModule->iVersion>=4 ); assert( pModule->xIntegrity!=0 ); sqlite3VtabLock(pTab->u.vtab.p); assert( pOp->p1>=0 && pOp->p1<db->nDb ); rc = pModule->xIntegrity(pVtab, db->aDb[pOp->p1].zDbSName, pTab->zName, pOp->p3, &zErr); sqlite3VtabUnlock(pTab->u.vtab.p); if( rc ){ sqlite3_free(zErr); goto abort_due_to_error; } if( zErr ){ sqlite3VdbeMemSetStr(pOut, zErr, -1, SQLITE_UTF8, sqlite3_free); } |
︙ | ︙ | |||
100445 100446 100447 100448 100449 100450 100451 100452 100453 100454 100455 100456 100457 100458 100459 100460 100461 100462 100463 100464 100465 100466 100467 100468 100469 100470 100471 100472 100473 100474 100475 | ** unused by OP_VColumn. */ case OP_VColumn: { /* ncycle */ sqlite3_vtab *pVtab; const sqlite3_module *pModule; Mem *pDest; sqlite3_context sContext; VdbeCursor *pCur = p->apCsr[pOp->p1]; assert( pCur!=0 ); assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) ); pDest = &aMem[pOp->p3]; memAboutToChange(p, pDest); if( pCur->nullRow ){ sqlite3VdbeMemSetNull(pDest); break; } assert( pCur->eCurType==CURTYPE_VTAB ); pVtab = pCur->uc.pVCur->pVtab; pModule = pVtab->pModule; assert( pModule->xColumn ); memset(&sContext, 0, sizeof(sContext)); sContext.pOut = pDest; sContext.enc = encoding; assert( pOp->p5==OPFLAG_NOCHNG || pOp->p5==0 ); if( pOp->p5 & OPFLAG_NOCHNG ){ sqlite3VdbeMemSetNull(pDest); pDest->flags = MEM_Null|MEM_Zero; pDest->u.nZero = 0; }else{ MemSetTypeFlag(pDest, MEM_Null); | > > > > | 100862 100863 100864 100865 100866 100867 100868 100869 100870 100871 100872 100873 100874 100875 100876 100877 100878 100879 100880 100881 100882 100883 100884 100885 100886 100887 100888 100889 100890 100891 100892 100893 100894 100895 100896 | ** unused by OP_VColumn. */ case OP_VColumn: { /* ncycle */ sqlite3_vtab *pVtab; const sqlite3_module *pModule; Mem *pDest; sqlite3_context sContext; FuncDef nullFunc; VdbeCursor *pCur = p->apCsr[pOp->p1]; assert( pCur!=0 ); assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) ); pDest = &aMem[pOp->p3]; memAboutToChange(p, pDest); if( pCur->nullRow ){ sqlite3VdbeMemSetNull(pDest); break; } assert( pCur->eCurType==CURTYPE_VTAB ); pVtab = pCur->uc.pVCur->pVtab; pModule = pVtab->pModule; assert( pModule->xColumn ); memset(&sContext, 0, sizeof(sContext)); sContext.pOut = pDest; sContext.enc = encoding; nullFunc.pUserData = 0; nullFunc.funcFlags = SQLITE_RESULT_SUBTYPE; sContext.pFunc = &nullFunc; assert( pOp->p5==OPFLAG_NOCHNG || pOp->p5==0 ); if( pOp->p5 & OPFLAG_NOCHNG ){ sqlite3VdbeMemSetNull(pDest); pDest->flags = MEM_Null|MEM_Zero; pDest->u.nZero = 0; }else{ MemSetTypeFlag(pDest, MEM_Null); |
︙ | ︙ | |||
100793 100794 100795 100796 100797 100798 100799 100800 100801 100802 100803 100804 100805 100806 | ** Clear the subtype from register P1. */ case OP_ClrSubtype: { /* in1 */ pIn1 = &aMem[pOp->p1]; pIn1->flags &= ~MEM_Subtype; break; } /* Opcode: FilterAdd P1 * P3 P4 * ** Synopsis: filter(P1) += key(P3@P4) ** ** Compute a hash on the P4 registers starting with r[P3] and ** add that hash to the bloom filter contained in r[P1]. */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 101214 101215 101216 101217 101218 101219 101220 101221 101222 101223 101224 101225 101226 101227 101228 101229 101230 101231 101232 101233 101234 101235 101236 101237 101238 101239 101240 101241 101242 101243 101244 101245 101246 101247 101248 101249 101250 101251 101252 101253 101254 101255 101256 101257 101258 101259 101260 101261 101262 101263 | ** Clear the subtype from register P1. */ case OP_ClrSubtype: { /* in1 */ pIn1 = &aMem[pOp->p1]; pIn1->flags &= ~MEM_Subtype; break; } /* Opcode: GetSubtype P1 P2 * * * ** Synopsis: r[P2] = r[P1].subtype ** ** Extract the subtype value from register P1 and write that subtype ** into register P2. If P1 has no subtype, then P1 gets a NULL. */ case OP_GetSubtype: { /* in1 out2 */ pIn1 = &aMem[pOp->p1]; pOut = &aMem[pOp->p2]; if( pIn1->flags & MEM_Subtype ){ sqlite3VdbeMemSetInt64(pOut, pIn1->eSubtype); }else{ sqlite3VdbeMemSetNull(pOut); } break; } /* Opcode: SetSubtype P1 P2 * * * ** Synopsis: r[P2].subtype = r[P1] ** ** Set the subtype value of register P2 to the integer from register P1. ** If P1 is NULL, clear the subtype from p2. */ case OP_SetSubtype: { /* in1 out2 */ pIn1 = &aMem[pOp->p1]; pOut = &aMem[pOp->p2]; if( pIn1->flags & MEM_Null ){ pOut->flags &= ~MEM_Subtype; }else{ assert( pIn1->flags & MEM_Int ); pOut->flags |= MEM_Subtype; pOut->eSubtype = (u8)(pIn1->u.i & 0xff); } break; } /* Opcode: FilterAdd P1 * P3 P4 * ** Synopsis: filter(P1) += key(P3@P4) ** ** Compute a hash on the P4 registers starting with r[P3] and ** add that hash to the bloom filter contained in r[P1]. */ |
︙ | ︙ | |||
105842 105843 105844 105845 105846 105847 105848 105849 105850 105851 105852 105853 105854 105855 | int n; Table *pExTab; n = pExpr->iColumn; assert( ExprUseYTab(pExpr) ); pExTab = pExpr->y.pTab; assert( pExTab!=0 ); if( (pExTab->tabFlags & TF_HasGenerated)!=0 && (pExTab->aCol[n].colFlags & COLFLAG_GENERATED)!=0 ){ testcase( pExTab->nCol==BMS-1 ); testcase( pExTab->nCol==BMS ); return pExTab->nCol>=BMS ? ALLBITS : MASKBIT(pExTab->nCol)-1; }else{ | > | 106299 106300 106301 106302 106303 106304 106305 106306 106307 106308 106309 106310 106311 106312 106313 | int n; Table *pExTab; n = pExpr->iColumn; assert( ExprUseYTab(pExpr) ); pExTab = pExpr->y.pTab; assert( pExTab!=0 ); assert( n < pExTab->nCol ); if( (pExTab->tabFlags & TF_HasGenerated)!=0 && (pExTab->aCol[n].colFlags & COLFLAG_GENERATED)!=0 ){ testcase( pExTab->nCol==BMS-1 ); testcase( pExTab->nCol==BMS ); return pExTab->nCol>=BMS ? ALLBITS : MASKBIT(pExTab->nCol)-1; }else{ |
︙ | ︙ | |||
106418 106419 106420 106421 106422 106423 106424 106425 106426 106427 106428 106429 106430 106431 | sqlite3ErrorMsg(pParse, "%s: %s.%s", zErr, zTab, zCol); }else{ sqlite3ErrorMsg(pParse, "%s: %s", zErr, zCol); } sqlite3RecordErrorOffsetOfExpr(pParse->db, pExpr); pParse->checkSchema = 1; pTopNC->nNcErr++; } assert( pFJMatch==0 ); /* Remove all substructure from pExpr */ if( !ExprHasProperty(pExpr,(EP_TokenOnly|EP_Leaf)) ){ sqlite3ExprDelete(db, pExpr->pLeft); pExpr->pLeft = 0; | > | 106876 106877 106878 106879 106880 106881 106882 106883 106884 106885 106886 106887 106888 106889 106890 | sqlite3ErrorMsg(pParse, "%s: %s.%s", zErr, zTab, zCol); }else{ sqlite3ErrorMsg(pParse, "%s: %s", zErr, zCol); } sqlite3RecordErrorOffsetOfExpr(pParse->db, pExpr); pParse->checkSchema = 1; pTopNC->nNcErr++; eNewExprOp = TK_NULL; } assert( pFJMatch==0 ); /* Remove all substructure from pExpr */ if( !ExprHasProperty(pExpr,(EP_TokenOnly|EP_Leaf)) ){ sqlite3ExprDelete(db, pExpr->pLeft); pExpr->pLeft = 0; |
︙ | ︙ | |||
106444 106445 106446 106447 106448 106449 106450 | ** if the mask contains extra set bits. However, it is important to ** avoid setting bits beyond the maximum column number of the table. ** (See ticket [b92e5e8ec2cdbaa1]). ** ** If a generated column is referenced, set bits for every column ** of the table. */ | | | 106903 106904 106905 106906 106907 106908 106909 106910 106911 106912 106913 106914 106915 106916 106917 | ** if the mask contains extra set bits. However, it is important to ** avoid setting bits beyond the maximum column number of the table. ** (See ticket [b92e5e8ec2cdbaa1]). ** ** If a generated column is referenced, set bits for every column ** of the table. */ if( pExpr->iColumn>=0 && cnt==1 && pMatch!=0 ){ pMatch->colUsed |= sqlite3ExprColUsed(pExpr); } pExpr->op = eNewExprOp; lookupname_end: if( cnt==1 ){ assert( pNC!=0 ); |
︙ | ︙ | |||
106909 106910 106911 106912 106913 106914 106915 | sqlite3WalkExpr(pWalker, pExpr->y.pWin->pFilter); } #endif pNC2 = pNC; while( pNC2 && sqlite3ReferencesSrcList(pParse, pExpr, pNC2->pSrcList)==0 ){ | | > | 107368 107369 107370 107371 107372 107373 107374 107375 107376 107377 107378 107379 107380 107381 107382 107383 107384 107385 107386 107387 | sqlite3WalkExpr(pWalker, pExpr->y.pWin->pFilter); } #endif pNC2 = pNC; while( pNC2 && sqlite3ReferencesSrcList(pParse, pExpr, pNC2->pSrcList)==0 ){ pExpr->op2 += (1 + pNC2->nNestedSelect); pNC2 = pNC2->pNext; } assert( pDef!=0 || IN_RENAME_OBJECT ); if( pNC2 && pDef ){ pExpr->op2 += pNC2->nNestedSelect; assert( SQLITE_FUNC_MINMAX==NC_MinMaxAgg ); assert( SQLITE_FUNC_ANYORDER==NC_OrderAgg ); testcase( (pDef->funcFlags & SQLITE_FUNC_MINMAX)!=0 ); testcase( (pDef->funcFlags & SQLITE_FUNC_ANYORDER)!=0 ); pNC2->ncFlags |= NC_HasAgg | ((pDef->funcFlags^SQLITE_FUNC_ANYORDER) & (SQLITE_FUNC_MINMAX|SQLITE_FUNC_ANYORDER)); |
︙ | ︙ | |||
107472 107473 107474 107475 107476 107477 107478 107479 107480 107481 107482 107483 107484 107485 | assert( pSub->pPrior && pSub->pOrderBy==0 ); pSub->pOrderBy = p->pOrderBy; p->pOrderBy = 0; } /* Recursively resolve names in all subqueries in the FROM clause */ for(i=0; i<p->pSrc->nSrc; i++){ SrcItem *pItem = &p->pSrc->a[i]; if( pItem->pSelect && (pItem->pSelect->selFlags & SF_Resolved)==0 ){ int nRef = pOuterNC ? pOuterNC->nRef : 0; const char *zSavedContext = pParse->zAuthContext; if( pItem->zName ) pParse->zAuthContext = pItem->zName; | > | 107932 107933 107934 107935 107936 107937 107938 107939 107940 107941 107942 107943 107944 107945 107946 | assert( pSub->pPrior && pSub->pOrderBy==0 ); pSub->pOrderBy = p->pOrderBy; p->pOrderBy = 0; } /* Recursively resolve names in all subqueries in the FROM clause */ if( pOuterNC ) pOuterNC->nNestedSelect++; for(i=0; i<p->pSrc->nSrc; i++){ SrcItem *pItem = &p->pSrc->a[i]; if( pItem->pSelect && (pItem->pSelect->selFlags & SF_Resolved)==0 ){ int nRef = pOuterNC ? pOuterNC->nRef : 0; const char *zSavedContext = pParse->zAuthContext; if( pItem->zName ) pParse->zAuthContext = pItem->zName; |
︙ | ︙ | |||
107495 107496 107497 107498 107499 107500 107501 107502 107503 107504 107505 107506 107507 107508 | ** the refcount on all contexts between the current one and the ** context containing the column when it resolves a name. */ if( pOuterNC ){ assert( pItem->fg.isCorrelated==0 && pOuterNC->nRef>=nRef ); pItem->fg.isCorrelated = (pOuterNC->nRef>nRef); } } } /* Set up the local name-context to pass to sqlite3ResolveExprNames() to ** resolve the result-set expression list. */ sNC.ncFlags = NC_AllowAgg|NC_AllowWin; sNC.pSrcList = p->pSrc; | > > > | 107956 107957 107958 107959 107960 107961 107962 107963 107964 107965 107966 107967 107968 107969 107970 107971 107972 | ** the refcount on all contexts between the current one and the ** context containing the column when it resolves a name. */ if( pOuterNC ){ assert( pItem->fg.isCorrelated==0 && pOuterNC->nRef>=nRef ); pItem->fg.isCorrelated = (pOuterNC->nRef>nRef); } } } if( pOuterNC && ALWAYS(pOuterNC->nNestedSelect>0) ){ pOuterNC->nNestedSelect--; } /* Set up the local name-context to pass to sqlite3ResolveExprNames() to ** resolve the result-set expression list. */ sNC.ncFlags = NC_AllowAgg|NC_AllowWin; sNC.pSrcList = p->pSrc; |
︙ | ︙ | |||
109083 109084 109085 109086 109087 109088 109089 | return; } assert( pExpr->op==TK_FUNCTION ); assert( pExpr->pLeft==0 ); assert( ExprUseXList(pExpr) ); if( pExpr->x.pList==0 || NEVER(pExpr->x.pList->nExpr==0) ){ /* Ignore ORDER BY on zero-argument aggregates */ | | < < | 109547 109548 109549 109550 109551 109552 109553 109554 109555 109556 109557 109558 109559 109560 109561 | return; } assert( pExpr->op==TK_FUNCTION ); assert( pExpr->pLeft==0 ); assert( ExprUseXList(pExpr) ); if( pExpr->x.pList==0 || NEVER(pExpr->x.pList->nExpr==0) ){ /* Ignore ORDER BY on zero-argument aggregates */ sqlite3ParserAddCleanup(pParse, sqlite3ExprListDeleteGeneric, pOrderBy); return; } if( IsWindowFunc(pExpr) ){ sqlite3ExprOrderByAggregateError(pParse, pExpr); sqlite3ExprListDelete(db, pOrderBy); return; } |
︙ | ︙ | |||
109266 109267 109268 109269 109270 109271 109272 109273 109274 109275 109276 109277 109278 109279 | if( !ExprHasProperty(p, EP_Static) ){ sqlite3DbNNFreeNN(db, p); } } SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){ if( p ) sqlite3ExprDeleteNN(db, p); } /* ** Clear both elements of an OnOrUsing object */ SQLITE_PRIVATE void sqlite3ClearOnOrUsing(sqlite3 *db, OnOrUsing *p){ if( p==0 ){ /* Nothing to clear */ | > > > | 109728 109729 109730 109731 109732 109733 109734 109735 109736 109737 109738 109739 109740 109741 109742 109743 109744 | if( !ExprHasProperty(p, EP_Static) ){ sqlite3DbNNFreeNN(db, p); } } SQLITE_PRIVATE void sqlite3ExprDelete(sqlite3 *db, Expr *p){ if( p ) sqlite3ExprDeleteNN(db, p); } SQLITE_PRIVATE void sqlite3ExprDeleteGeneric(sqlite3 *db, void *p){ if( ALWAYS(p) ) sqlite3ExprDeleteNN(db, (Expr*)p); } /* ** Clear both elements of an OnOrUsing object */ SQLITE_PRIVATE void sqlite3ClearOnOrUsing(sqlite3 *db, OnOrUsing *p){ if( p==0 ){ /* Nothing to clear */ |
︙ | ︙ | |||
109291 109292 109293 109294 109295 109296 109297 | ** ** The pExpr might be deleted immediately on an OOM error. ** ** The deferred delete is (currently) implemented by adding the ** pExpr to the pParse->pConstExpr list with a register number of 0. */ SQLITE_PRIVATE void sqlite3ExprDeferredDelete(Parse *pParse, Expr *pExpr){ | | < < | 109756 109757 109758 109759 109760 109761 109762 109763 109764 109765 109766 109767 109768 109769 109770 | ** ** The pExpr might be deleted immediately on an OOM error. ** ** The deferred delete is (currently) implemented by adding the ** pExpr to the pParse->pConstExpr list with a register number of 0. */ SQLITE_PRIVATE void sqlite3ExprDeferredDelete(Parse *pParse, Expr *pExpr){ sqlite3ParserAddCleanup(pParse, sqlite3ExprDeleteGeneric, pExpr); } /* Invoke sqlite3RenameExprUnmap() and sqlite3ExprDelete() on the ** expression. */ SQLITE_PRIVATE void sqlite3ExprUnmapAndDelete(Parse *pParse, Expr *p){ if( p ){ |
︙ | ︙ | |||
109498 109499 109500 109501 109502 109503 109504 | } if( dupFlags ){ assert( (int)(sEdupBuf.zEnd - sEdupBuf.zAlloc) >= nNewSize+nToken ); assert( ExprHasProperty(p, EP_Reduced)==0 ); memcpy(sEdupBuf.zAlloc, p, nNewSize); }else{ u32 nSize = (u32)exprStructSize(p); | | > | 109961 109962 109963 109964 109965 109966 109967 109968 109969 109970 109971 109972 109973 109974 109975 109976 | } if( dupFlags ){ assert( (int)(sEdupBuf.zEnd - sEdupBuf.zAlloc) >= nNewSize+nToken ); assert( ExprHasProperty(p, EP_Reduced)==0 ); memcpy(sEdupBuf.zAlloc, p, nNewSize); }else{ u32 nSize = (u32)exprStructSize(p); assert( (int)(sEdupBuf.zEnd - sEdupBuf.zAlloc) >= (int)EXPR_FULLSIZE+nToken ); memcpy(sEdupBuf.zAlloc, p, nSize); if( nSize<EXPR_FULLSIZE ){ memset(&sEdupBuf.zAlloc[nSize], 0, EXPR_FULLSIZE-nSize); } nNewSize = EXPR_FULLSIZE; } |
︙ | ︙ | |||
110098 110099 110100 110101 110102 110103 110104 110105 110106 110107 110108 110109 110110 110111 | pItem++; }while( --i>0 ); sqlite3DbNNFreeNN(db, pList); } SQLITE_PRIVATE void sqlite3ExprListDelete(sqlite3 *db, ExprList *pList){ if( pList ) exprListDeleteNN(db, pList); } /* ** Return the bitwise-OR of all Expr.flags fields in the given ** ExprList. */ SQLITE_PRIVATE u32 sqlite3ExprListFlags(const ExprList *pList){ int i; | > > > | 110562 110563 110564 110565 110566 110567 110568 110569 110570 110571 110572 110573 110574 110575 110576 110577 110578 | pItem++; }while( --i>0 ); sqlite3DbNNFreeNN(db, pList); } SQLITE_PRIVATE void sqlite3ExprListDelete(sqlite3 *db, ExprList *pList){ if( pList ) exprListDeleteNN(db, pList); } SQLITE_PRIVATE void sqlite3ExprListDeleteGeneric(sqlite3 *db, void *pList){ if( ALWAYS(pList) ) exprListDeleteNN(db, (ExprList*)pList); } /* ** Return the bitwise-OR of all Expr.flags fields in the given ** ExprList. */ SQLITE_PRIVATE u32 sqlite3ExprListFlags(const ExprList *pList){ int i; |
︙ | ︙ | |||
110597 110598 110599 110600 110601 110602 110603 | case TK_STRING: case TK_FLOAT: case TK_BLOB: return 0; case TK_COLUMN: assert( ExprUseYTab(p) ); return ExprHasProperty(p, EP_CanBeNull) || | | > | 111064 111065 111066 111067 111068 111069 111070 111071 111072 111073 111074 111075 111076 111077 111078 111079 111080 111081 | case TK_STRING: case TK_FLOAT: case TK_BLOB: return 0; case TK_COLUMN: assert( ExprUseYTab(p) ); return ExprHasProperty(p, EP_CanBeNull) || NEVER(p->y.pTab==0) || /* Reference to column of index on expr */ (p->iColumn>=0 && p->y.pTab->aCol!=0 /* Possible due to prior error */ && ALWAYS(p->iColumn<p->y.pTab->nCol) && p->y.pTab->aCol[p->iColumn].notNull==0); default: return 1; } } /* |
︙ | ︙ | |||
113181 113182 113183 113184 113185 113186 113187 | assert( pExpr==0 || !ExprHasVVAProperty(pExpr,EP_Immutable) ); assert( target>0 && target<=pParse->nMem ); assert( pParse->pVdbe!=0 || pParse->db->mallocFailed ); if( pParse->pVdbe==0 ) return; inReg = sqlite3ExprCodeTarget(pParse, pExpr, target); if( inReg!=target ){ u8 op; | > > | | | 113649 113650 113651 113652 113653 113654 113655 113656 113657 113658 113659 113660 113661 113662 113663 113664 113665 113666 | assert( pExpr==0 || !ExprHasVVAProperty(pExpr,EP_Immutable) ); assert( target>0 && target<=pParse->nMem ); assert( pParse->pVdbe!=0 || pParse->db->mallocFailed ); if( pParse->pVdbe==0 ) return; inReg = sqlite3ExprCodeTarget(pParse, pExpr, target); if( inReg!=target ){ u8 op; Expr *pX = sqlite3ExprSkipCollateAndLikely(pExpr); testcase( pX!=pExpr ); if( ALWAYS(pX) && (ExprHasProperty(pX,EP_Subquery) || pX->op==TK_REGISTER) ){ op = OP_Copy; }else{ op = OP_SCopy; } sqlite3VdbeAddOp2(pParse->pVdbe, op, inReg, target); } |
︙ | ︙ | |||
113902 113903 113904 113905 113906 113907 113908 | /* ** Like sqlite3ExprCompare() except COLLATE operators at the top-level ** are ignored. */ SQLITE_PRIVATE int sqlite3ExprCompareSkip(Expr *pA,Expr *pB, int iTab){ return sqlite3ExprCompare(0, | | | | 114372 114373 114374 114375 114376 114377 114378 114379 114380 114381 114382 114383 114384 114385 114386 114387 | /* ** Like sqlite3ExprCompare() except COLLATE operators at the top-level ** are ignored. */ SQLITE_PRIVATE int sqlite3ExprCompareSkip(Expr *pA,Expr *pB, int iTab){ return sqlite3ExprCompare(0, sqlite3ExprSkipCollate(pA), sqlite3ExprSkipCollate(pB), iTab); } /* ** Return non-zero if Expr p can only be true if pNN is not NULL. ** ** Or if seenNot is true, return non-zero if Expr p can only be |
︙ | ︙ | |||
114628 114629 114630 114631 114632 114633 114634 114635 114636 114637 114638 114639 114640 | } /* end loop over pSrcList */ } return WRC_Continue; } case TK_AGG_FUNCTION: { if( (pNC->ncFlags & NC_InAggFunc)==0 && pWalker->walkerDepth==pExpr->op2 ){ /* Check to see if pExpr is a duplicate of another aggregate ** function that is already in the pAggInfo structure */ struct AggInfo_func *pItem = pAggInfo->aFunc; for(i=0; i<pAggInfo->nFunc; i++, pItem++){ | > | | 115098 115099 115100 115101 115102 115103 115104 115105 115106 115107 115108 115109 115110 115111 115112 115113 115114 115115 115116 115117 115118 115119 | } /* end loop over pSrcList */ } return WRC_Continue; } case TK_AGG_FUNCTION: { if( (pNC->ncFlags & NC_InAggFunc)==0 && pWalker->walkerDepth==pExpr->op2 && pExpr->pAggInfo==0 ){ /* Check to see if pExpr is a duplicate of another aggregate ** function that is already in the pAggInfo structure */ struct AggInfo_func *pItem = pAggInfo->aFunc; for(i=0; i<pAggInfo->nFunc; i++, pItem++){ if( NEVER(pItem->pFExpr==pExpr) ) break; if( sqlite3ExprCompare(0, pItem->pFExpr, pExpr, -1)==0 ){ break; } } if( i>=pAggInfo->nFunc ){ /* pExpr is original. Make a new entry in pAggInfo->aFunc[] */ |
︙ | ︙ | |||
114666 114667 114668 114669 114670 114671 114672 114673 114674 114675 114676 114677 114678 114679 114680 114681 | ExprList *pOBList; assert( nArg>0 ); assert( pExpr->pLeft->op==TK_ORDER ); assert( ExprUseXList(pExpr->pLeft) ); pItem->iOBTab = pParse->nTab++; pOBList = pExpr->pLeft->x.pList; assert( pOBList->nExpr>0 ); if( pOBList->nExpr==1 && nArg==1 && sqlite3ExprCompare(0,pOBList->a[0].pExpr, pExpr->x.pList->a[0].pExpr,0)==0 ){ pItem->bOBPayload = 0; }else{ pItem->bOBPayload = 1; } | > > | > | 115137 115138 115139 115140 115141 115142 115143 115144 115145 115146 115147 115148 115149 115150 115151 115152 115153 115154 115155 115156 115157 115158 115159 115160 115161 115162 115163 | ExprList *pOBList; assert( nArg>0 ); assert( pExpr->pLeft->op==TK_ORDER ); assert( ExprUseXList(pExpr->pLeft) ); pItem->iOBTab = pParse->nTab++; pOBList = pExpr->pLeft->x.pList; assert( pOBList->nExpr>0 ); assert( pItem->bOBUnique==0 ); if( pOBList->nExpr==1 && nArg==1 && sqlite3ExprCompare(0,pOBList->a[0].pExpr, pExpr->x.pList->a[0].pExpr,0)==0 ){ pItem->bOBPayload = 0; pItem->bOBUnique = ExprHasProperty(pExpr, EP_Distinct); }else{ pItem->bOBPayload = 1; } pItem->bUseSubtype = (pItem->pFunc->funcFlags & SQLITE_SUBTYPE)!=0; }else{ pItem->iOBTab = -1; } if( ExprHasProperty(pExpr, EP_Distinct) && !pItem->bOBUnique ){ pItem->iDistinct = pParse->nTab++; }else{ pItem->iDistinct = -1; |
︙ | ︙ | |||
117442 117443 117444 117445 117446 117447 117448 | ** Three SQL functions - stat_init(), stat_push(), and stat_get() - ** share an instance of the following structure to hold their state ** information. */ typedef struct StatAccum StatAccum; typedef struct StatSample StatSample; struct StatSample { | < > | 117916 117917 117918 117919 117920 117921 117922 117923 117924 117925 117926 117927 117928 117929 117930 117931 117932 | ** Three SQL functions - stat_init(), stat_push(), and stat_get() - ** share an instance of the following structure to hold their state ** information. */ typedef struct StatAccum StatAccum; typedef struct StatSample StatSample; struct StatSample { tRowcnt *anDLt; /* sqlite_stat4.nDLt */ #ifdef SQLITE_ENABLE_STAT4 tRowcnt *anEq; /* sqlite_stat4.nEq */ tRowcnt *anLt; /* sqlite_stat4.nLt */ union { i64 iRowid; /* Rowid in main table of the key */ u8 *aRowid; /* Key for WITHOUT ROWID tables */ } u; u32 nRowid; /* Sizeof aRowid[] */ u8 isPSample; /* True if a periodic sample */ |
︙ | ︙ | |||
117602 117603 117604 117605 117606 117607 117608 | nColUp = sizeof(tRowcnt)<8 ? (nCol+1)&~1 : nCol; nKeyCol = sqlite3_value_int(argv[1]); assert( nKeyCol<=nCol ); assert( nKeyCol>0 ); /* Allocate the space required for the StatAccum object */ n = sizeof(*p) | < | > < > | 118076 118077 118078 118079 118080 118081 118082 118083 118084 118085 118086 118087 118088 118089 118090 118091 118092 118093 118094 118095 118096 118097 118098 118099 118100 118101 118102 118103 118104 118105 118106 118107 118108 118109 118110 118111 118112 118113 118114 118115 | nColUp = sizeof(tRowcnt)<8 ? (nCol+1)&~1 : nCol; nKeyCol = sqlite3_value_int(argv[1]); assert( nKeyCol<=nCol ); assert( nKeyCol>0 ); /* Allocate the space required for the StatAccum object */ n = sizeof(*p) + sizeof(tRowcnt)*nColUp; /* StatAccum.anDLt */ #ifdef SQLITE_ENABLE_STAT4 n += sizeof(tRowcnt)*nColUp; /* StatAccum.anEq */ if( mxSample ){ n += sizeof(tRowcnt)*nColUp /* StatAccum.anLt */ + sizeof(StatSample)*(nCol+mxSample) /* StatAccum.aBest[], a[] */ + sizeof(tRowcnt)*3*nColUp*(nCol+mxSample); } #endif p = sqlite3DbMallocZero(db, n); if( p==0 ){ sqlite3_result_error_nomem(context); return; } p->db = db; p->nEst = sqlite3_value_int64(argv[2]); p->nRow = 0; p->nLimit = sqlite3_value_int64(argv[3]); p->nCol = nCol; p->nKeyCol = nKeyCol; p->nSkipAhead = 0; p->current.anDLt = (tRowcnt*)&p[1]; #ifdef SQLITE_ENABLE_STAT4 p->current.anEq = &p->current.anDLt[nColUp]; p->mxSample = p->nLimit==0 ? mxSample : 0; if( mxSample ){ u8 *pSpace; /* Allocated space not yet assigned */ int i; /* Used to iterate through p->aSample[] */ p->iGet = -1; p->nPSample = (tRowcnt)(p->nEst/(mxSample/3+1) + 1); |
︙ | ︙ | |||
117894 117895 117896 117897 117898 117899 117900 117901 117902 117903 117904 117905 117906 117907 117908 117909 117910 117911 117912 117913 117914 117915 117916 | UNUSED_PARAMETER( argc ); UNUSED_PARAMETER( context ); assert( p->nCol>0 ); assert( iChng<p->nCol ); if( p->nRow==0 ){ /* This is the first call to this function. Do initialization. */ for(i=0; i<p->nCol; i++) p->current.anEq[i] = 1; }else{ /* Second and subsequent calls get processed here */ #ifdef SQLITE_ENABLE_STAT4 if( p->mxSample ) samplePushPrevious(p, iChng); #endif /* Update anDLt[], anLt[] and anEq[] to reflect the values that apply ** to the current row of the index. */ for(i=0; i<iChng; i++){ p->current.anEq[i]++; } for(i=iChng; i<p->nCol; i++){ p->current.anDLt[i]++; #ifdef SQLITE_ENABLE_STAT4 if( p->mxSample ) p->current.anLt[i] += p->current.anEq[i]; | > > > > < > | 118368 118369 118370 118371 118372 118373 118374 118375 118376 118377 118378 118379 118380 118381 118382 118383 118384 118385 118386 118387 118388 118389 118390 118391 118392 118393 118394 118395 118396 118397 118398 118399 118400 118401 118402 118403 | UNUSED_PARAMETER( argc ); UNUSED_PARAMETER( context ); assert( p->nCol>0 ); assert( iChng<p->nCol ); if( p->nRow==0 ){ /* This is the first call to this function. Do initialization. */ #ifdef SQLITE_ENABLE_STAT4 for(i=0; i<p->nCol; i++) p->current.anEq[i] = 1; #endif }else{ /* Second and subsequent calls get processed here */ #ifdef SQLITE_ENABLE_STAT4 if( p->mxSample ) samplePushPrevious(p, iChng); #endif /* Update anDLt[], anLt[] and anEq[] to reflect the values that apply ** to the current row of the index. */ #ifdef SQLITE_ENABLE_STAT4 for(i=0; i<iChng; i++){ p->current.anEq[i]++; } #endif for(i=iChng; i<p->nCol; i++){ p->current.anDLt[i]++; #ifdef SQLITE_ENABLE_STAT4 if( p->mxSample ) p->current.anLt[i] += p->current.anEq[i]; p->current.anEq[i] = 1; #endif } } p->nRow++; #ifdef SQLITE_ENABLE_STAT4 if( p->mxSample ){ tRowcnt nLt; |
︙ | ︙ | |||
118045 118046 118047 118048 118049 118050 118051 118052 118053 118054 118055 118056 118057 118058 118059 | sqlite3_str_appendf(&sStat, "%llu", p->nSkipAhead ? (u64)p->nEst : (u64)p->nRow); for(i=0; i<p->nKeyCol; i++){ u64 nDistinct = p->current.anDLt[i] + 1; u64 iVal = (p->nRow + nDistinct - 1) / nDistinct; if( iVal==2 && p->nRow*10 <= nDistinct*11 ) iVal = 1; sqlite3_str_appendf(&sStat, " %llu", iVal); assert( p->current.anEq[i] ); } sqlite3ResultStrAccum(context, &sStat); } #ifdef SQLITE_ENABLE_STAT4 else if( eCall==STAT_GET_ROWID ){ if( p->iGet<0 ){ samplePushPrevious(p, 0); | > > | 118523 118524 118525 118526 118527 118528 118529 118530 118531 118532 118533 118534 118535 118536 118537 118538 118539 | sqlite3_str_appendf(&sStat, "%llu", p->nSkipAhead ? (u64)p->nEst : (u64)p->nRow); for(i=0; i<p->nKeyCol; i++){ u64 nDistinct = p->current.anDLt[i] + 1; u64 iVal = (p->nRow + nDistinct - 1) / nDistinct; if( iVal==2 && p->nRow*10 <= nDistinct*11 ) iVal = 1; sqlite3_str_appendf(&sStat, " %llu", iVal); #ifdef SQLITE_ENABLE_STAT4 assert( p->current.anEq[i] ); #endif } sqlite3ResultStrAccum(context, &sStat); } #ifdef SQLITE_ENABLE_STAT4 else if( eCall==STAT_GET_ROWID ){ if( p->iGet<0 ){ samplePushPrevious(p, 0); |
︙ | ︙ | |||
118734 118735 118736 118737 118738 118739 118740 118741 118742 118743 118744 118745 118746 118747 | else if( sqlite3_strglob("costmult=[0-9]*",z)==0 ){ pIndex->pTable->costMult = sqlite3LogEst(sqlite3Atoi(z+9)); } #endif while( z[0]!=0 && z[0]!=' ' ) z++; while( z[0]==' ' ) z++; } } } /* ** This callback is invoked once for each index when reading the ** sqlite_stat1 table. ** | > > > > > > > > > > | 119214 119215 119216 119217 119218 119219 119220 119221 119222 119223 119224 119225 119226 119227 119228 119229 119230 119231 119232 119233 119234 119235 119236 119237 | else if( sqlite3_strglob("costmult=[0-9]*",z)==0 ){ pIndex->pTable->costMult = sqlite3LogEst(sqlite3Atoi(z+9)); } #endif while( z[0]!=0 && z[0]!=' ' ) z++; while( z[0]==' ' ) z++; } /* Set the bLowQual flag if the peak number of rows obtained ** from a full equality match is so large that a full table scan ** seems likely to be faster than using the index. */ if( aLog[0] > 66 /* Index has more than 100 rows */ && aLog[0] <= aLog[nOut-1] /* And only a single value seen */ ){ pIndex->bLowQual = 1; } } } /* ** This callback is invoked once for each index when reading the ** sqlite_stat1 table. ** |
︙ | ︙ | |||
120780 120781 120782 120783 120784 120785 120786 | /* ** Return the expression associated with a column. The expression might be ** the DEFAULT clause or the AS clause of a generated column. ** Return NULL if the column has no associated expression. */ SQLITE_PRIVATE Expr *sqlite3ColumnExpr(Table *pTab, Column *pCol){ if( pCol->iDflt==0 ) return 0; | | | 121270 121271 121272 121273 121274 121275 121276 121277 121278 121279 121280 121281 121282 121283 121284 | /* ** Return the expression associated with a column. The expression might be ** the DEFAULT clause or the AS clause of a generated column. ** Return NULL if the column has no associated expression. */ SQLITE_PRIVATE Expr *sqlite3ColumnExpr(Table *pTab, Column *pCol){ if( pCol->iDflt==0 ) return 0; if( !IsOrdinaryTable(pTab) ) return 0; if( NEVER(pTab->u.tab.pDfltList==0) ) return 0; if( NEVER(pTab->u.tab.pDfltList->nExpr<pCol->iDflt) ) return 0; return pTab->u.tab.pDfltList->a[pCol->iDflt-1].pExpr; } /* ** Set the collating sequence name for a column. |
︙ | ︙ | |||
120932 120933 120934 120935 120936 120937 120938 120939 120940 120941 120942 120943 120944 120945 | } SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3 *db, Table *pTable){ /* Do not delete the table until the reference count reaches zero. */ assert( db!=0 ); if( !pTable ) return; if( db->pnBytesFreed==0 && (--pTable->nTabRef)>0 ) return; deleteTable(db, pTable); } /* ** Unlink the given table from the hash tables and the delete the ** table structure with all its indices and foreign keys. */ | > > > | 121422 121423 121424 121425 121426 121427 121428 121429 121430 121431 121432 121433 121434 121435 121436 121437 121438 | } SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3 *db, Table *pTable){ /* Do not delete the table until the reference count reaches zero. */ assert( db!=0 ); if( !pTable ) return; if( db->pnBytesFreed==0 && (--pTable->nTabRef)>0 ) return; deleteTable(db, pTable); } SQLITE_PRIVATE void sqlite3DeleteTableGeneric(sqlite3 *db, void *pTable){ sqlite3DeleteTable(db, (Table*)pTable); } /* ** Unlink the given table from the hash tables and the delete the ** table structure with all its indices and foreign keys. */ |
︙ | ︙ | |||
121467 121468 121469 121470 121471 121472 121473 | if( pTab ) pTab->tabFlags |= TF_HasHidden; }else if( pTab && pCol!=pTab->aCol && (pCol[-1].colFlags & COLFLAG_HIDDEN) ){ pTab->tabFlags |= TF_OOOHidden; } } #endif | < < < < < < < | > | | 121960 121961 121962 121963 121964 121965 121966 121967 121968 121969 121970 121971 121972 121973 121974 121975 121976 121977 121978 121979 121980 121981 | if( pTab ) pTab->tabFlags |= TF_HasHidden; }else if( pTab && pCol!=pTab->aCol && (pCol[-1].colFlags & COLFLAG_HIDDEN) ){ pTab->tabFlags |= TF_OOOHidden; } } #endif /* ** Clean up the data structures associated with the RETURNING clause. */ static void sqlite3DeleteReturning(sqlite3 *db, void *pArg){ Returning *pRet = (Returning*)pArg; Hash *pHash; pHash = &(db->aDb[1].pSchema->trigHash); sqlite3HashInsert(pHash, pRet->zName, 0); sqlite3ExprListDelete(db, pRet->pReturnEL); sqlite3DbFree(db, pRet); } /* ** Add the RETURNING clause to the parse currently underway. ** |
︙ | ︙ | |||
121519 121520 121521 121522 121523 121524 121525 | if( pRet==0 ){ sqlite3ExprListDelete(db, pList); return; } pParse->u1.pReturning = pRet; pRet->pParse = pParse; pRet->pReturnEL = pList; | | < > > | | | | 122006 122007 122008 122009 122010 122011 122012 122013 122014 122015 122016 122017 122018 122019 122020 122021 122022 122023 122024 122025 122026 122027 122028 122029 122030 122031 122032 122033 122034 122035 122036 122037 122038 | if( pRet==0 ){ sqlite3ExprListDelete(db, pList); return; } pParse->u1.pReturning = pRet; pRet->pParse = pParse; pRet->pReturnEL = pList; sqlite3ParserAddCleanup(pParse, sqlite3DeleteReturning, pRet); testcase( pParse->earlyCleanup ); if( db->mallocFailed ) return; sqlite3_snprintf(sizeof(pRet->zName), pRet->zName, "sqlite_returning_%p", pParse); pRet->retTrig.zName = pRet->zName; pRet->retTrig.op = TK_RETURNING; pRet->retTrig.tr_tm = TRIGGER_AFTER; pRet->retTrig.bReturning = 1; pRet->retTrig.pSchema = db->aDb[1].pSchema; pRet->retTrig.pTabSchema = db->aDb[1].pSchema; pRet->retTrig.step_list = &pRet->retTStep; pRet->retTStep.op = TK_RETURNING; pRet->retTStep.pTrig = &pRet->retTrig; pRet->retTStep.pExprList = pList; pHash = &(db->aDb[1].pSchema->trigHash); assert( sqlite3HashFind(pHash, pRet->zName)==0 || pParse->nErr || pParse->ifNotExists ); if( sqlite3HashInsert(pHash, pRet->zName, &pRet->retTrig) ==&pRet->retTrig ){ sqlite3OomFault(db); } } /* ** Add a new column to the table currently being constructed. |
︙ | ︙ | |||
121717 121718 121719 121720 121721 121722 121723 | SQLITE_PRIVATE char sqlite3AffinityType(const char *zIn, Column *pCol){ u32 h = 0; char aff = SQLITE_AFF_NUMERIC; const char *zChar = 0; assert( zIn!=0 ); while( zIn[0] ){ | > | | 122205 122206 122207 122208 122209 122210 122211 122212 122213 122214 122215 122216 122217 122218 122219 122220 | SQLITE_PRIVATE char sqlite3AffinityType(const char *zIn, Column *pCol){ u32 h = 0; char aff = SQLITE_AFF_NUMERIC; const char *zChar = 0; assert( zIn!=0 ); while( zIn[0] ){ u8 x = *(u8*)zIn; h = (h<<8) + sqlite3UpperToLower[x]; zIn++; if( h==(('c'<<24)+('h'<<16)+('a'<<8)+'r') ){ /* CHAR */ aff = SQLITE_AFF_TEXT; zChar = zIn; }else if( h==(('c'<<24)+('l'<<16)+('o'<<8)+'b') ){ /* CLOB */ aff = SQLITE_AFF_TEXT; }else if( h==(('t'<<24)+('e'<<16)+('x'<<8)+'t') ){ /* TEXT */ |
︙ | ︙ | |||
125582 125583 125584 125585 125586 125587 125588 | } sqlite3DbFree(db, zColl); } iDb = sqlite3TwoPartName(pParse, pName1, pName2, &pObjName); if( iDb<0 ) return; z = sqlite3NameFromToken(db, pObjName); if( z==0 ) return; | | > | 126071 126072 126073 126074 126075 126076 126077 126078 126079 126080 126081 126082 126083 126084 126085 126086 126087 126088 126089 126090 126091 126092 126093 126094 126095 | } sqlite3DbFree(db, zColl); } iDb = sqlite3TwoPartName(pParse, pName1, pName2, &pObjName); if( iDb<0 ) return; z = sqlite3NameFromToken(db, pObjName); if( z==0 ) return; zDb = pName2->n ? db->aDb[iDb].zDbSName : 0; pTab = sqlite3FindTable(db, z, zDb); if( pTab ){ reindexTable(pParse, pTab, 0); sqlite3DbFree(db, z); return; } pIndex = sqlite3FindIndex(db, z, zDb); sqlite3DbFree(db, z); if( pIndex ){ iDb = sqlite3SchemaToIndex(db, pIndex->pTable->pSchema); sqlite3BeginWriteOperation(pParse, 0, iDb); sqlite3RefillIndex(pParse, pIndex, -1); return; } sqlite3ErrorMsg(pParse, "unable to identify the object to be reindexed"); } #endif |
︙ | ︙ | |||
125756 125757 125758 125759 125760 125761 125762 125763 125764 125765 125766 125767 125768 125769 | if( pWith ){ int i; for(i=0; i<pWith->nCte; i++){ cteClear(db, &pWith->a[i]); } sqlite3DbFree(db, pWith); } } #endif /* !defined(SQLITE_OMIT_CTE) */ /************** End of build.c ***********************************************/ /************** Begin file callback.c ****************************************/ /* ** 2005 May 23 | > > > | 126246 126247 126248 126249 126250 126251 126252 126253 126254 126255 126256 126257 126258 126259 126260 126261 126262 | if( pWith ){ int i; for(i=0; i<pWith->nCte; i++){ cteClear(db, &pWith->a[i]); } sqlite3DbFree(db, pWith); } } SQLITE_PRIVATE void sqlite3WithDeleteGeneric(sqlite3 *db, void *pWith){ sqlite3WithDelete(db, (With*)pWith); } #endif /* !defined(SQLITE_OMIT_CTE) */ /************** End of build.c ***********************************************/ /************** Begin file callback.c ****************************************/ /* ** 2005 May 23 |
︙ | ︙ | |||
135275 135276 135277 135278 135279 135280 135281 135282 135283 135284 135285 135286 135287 135288 | const char *(*db_name)(sqlite3*,int); /* Version 3.40.0 and later */ int (*value_encoding)(sqlite3_value*); /* Version 3.41.0 and later */ int (*is_interrupted)(sqlite3*); /* Version 3.43.0 and later */ int (*stmt_explain)(sqlite3_stmt*,int); }; /* ** This is the function signature used for all extension entry points. It ** is also defined in the file "loadext.c". */ typedef int (*sqlite3_loadext_entry)( | > > > | 135768 135769 135770 135771 135772 135773 135774 135775 135776 135777 135778 135779 135780 135781 135782 135783 135784 | const char *(*db_name)(sqlite3*,int); /* Version 3.40.0 and later */ int (*value_encoding)(sqlite3_value*); /* Version 3.41.0 and later */ int (*is_interrupted)(sqlite3*); /* Version 3.43.0 and later */ int (*stmt_explain)(sqlite3_stmt*,int); /* Version 3.44.0 and later */ void *(*get_clientdata)(sqlite3*,const char*); int (*set_clientdata)(sqlite3*, const char*, void*, void(*)(void*)); }; /* ** This is the function signature used for all extension entry points. It ** is also defined in the file "loadext.c". */ typedef int (*sqlite3_loadext_entry)( |
︙ | ︙ | |||
135605 135606 135607 135608 135609 135610 135611 135612 135613 135614 135615 135616 135617 135618 | #define sqlite3_db_name sqlite3_api->db_name /* Version 3.40.0 and later */ #define sqlite3_value_encoding sqlite3_api->value_encoding /* Version 3.41.0 and later */ #define sqlite3_is_interrupted sqlite3_api->is_interrupted /* Version 3.43.0 and later */ #define sqlite3_stmt_explain sqlite3_api->stmt_explain #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */ #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) /* This case when the file really is being compiled as a loadable ** extension */ # define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api=0; # define SQLITE_EXTENSION_INIT2(v) sqlite3_api=v; | > > > | 136101 136102 136103 136104 136105 136106 136107 136108 136109 136110 136111 136112 136113 136114 136115 136116 136117 | #define sqlite3_db_name sqlite3_api->db_name /* Version 3.40.0 and later */ #define sqlite3_value_encoding sqlite3_api->value_encoding /* Version 3.41.0 and later */ #define sqlite3_is_interrupted sqlite3_api->is_interrupted /* Version 3.43.0 and later */ #define sqlite3_stmt_explain sqlite3_api->stmt_explain /* Version 3.44.0 and later */ #define sqlite3_get_clientdata sqlite3_api->get_clientdata #define sqlite3_set_clientdata sqlite3_api->set_clientdata #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */ #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) /* This case when the file really is being compiled as a loadable ** extension */ # define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api=0; # define SQLITE_EXTENSION_INIT2(v) sqlite3_api=v; |
︙ | ︙ | |||
136123 136124 136125 136126 136127 136128 136129 | #endif sqlite3_db_name, /* Version 3.40.0 and later */ sqlite3_value_encoding, /* Version 3.41.0 and later */ sqlite3_is_interrupted, /* Version 3.43.0 and later */ | | > > > | 136622 136623 136624 136625 136626 136627 136628 136629 136630 136631 136632 136633 136634 136635 136636 136637 136638 136639 | #endif sqlite3_db_name, /* Version 3.40.0 and later */ sqlite3_value_encoding, /* Version 3.41.0 and later */ sqlite3_is_interrupted, /* Version 3.43.0 and later */ sqlite3_stmt_explain, /* Version 3.44.0 and later */ sqlite3_get_clientdata, sqlite3_set_clientdata }; /* True if x is the directory separator character */ #if SQLITE_OS_WIN # define DirSep(X) ((X)=='/'||(X)=='\\') #else |
︙ | ︙ | |||
138972 138973 138974 138975 138976 138977 138978 | sqlite3ViewGetColumnNames(pParse, pTab); if( pTab->u.vtab.p==0 ) continue; pVTab = pTab->u.vtab.p->pVtab; if( NEVER(pVTab==0) ) continue; if( NEVER(pVTab->pModule==0) ) continue; if( pVTab->pModule->iVersion<4 ) continue; if( pVTab->pModule->xIntegrity==0 ) continue; | | > | | 139474 139475 139476 139477 139478 139479 139480 139481 139482 139483 139484 139485 139486 139487 139488 139489 139490 | sqlite3ViewGetColumnNames(pParse, pTab); if( pTab->u.vtab.p==0 ) continue; pVTab = pTab->u.vtab.p->pVtab; if( NEVER(pVTab==0) ) continue; if( NEVER(pVTab->pModule==0) ) continue; if( pVTab->pModule->iVersion<4 ) continue; if( pVTab->pModule->xIntegrity==0 ) continue; sqlite3VdbeAddOp3(v, OP_VCheck, i, 3, isQuick); pTab->nTabRef++; sqlite3VdbeAppendP4(v, pTab, P4_TABLEREF); a1 = sqlite3VdbeAddOp1(v, OP_IsNull, 3); VdbeCoverage(v); integrityCheckResultRow(v); sqlite3VdbeJumpHere(v, a1); #endif continue; } if( isQuick || HasRowid(pTab) ){ |
︙ | ︙ | |||
141000 141001 141002 141003 141004 141005 141006 141007 141008 141009 141010 141011 141012 141013 | }while( (rc==SQLITE_ERROR_RETRY && (cnt++)<SQLITE_MAX_PREPARE_RETRY) || (rc==SQLITE_SCHEMA && (sqlite3ResetOneSchema(db,-1), cnt++)==0) ); sqlite3BtreeLeaveAll(db); rc = sqlite3ApiExit(db, rc); assert( (rc&db->errMask)==rc ); db->busyHandler.nBusy = 0; sqlite3_mutex_leave(db->mutex); return rc; } /* ** Rerun the compilation of a statement after a schema change. ** | > | 141503 141504 141505 141506 141507 141508 141509 141510 141511 141512 141513 141514 141515 141516 141517 | }while( (rc==SQLITE_ERROR_RETRY && (cnt++)<SQLITE_MAX_PREPARE_RETRY) || (rc==SQLITE_SCHEMA && (sqlite3ResetOneSchema(db,-1), cnt++)==0) ); sqlite3BtreeLeaveAll(db); rc = sqlite3ApiExit(db, rc); assert( (rc&db->errMask)==rc ); db->busyHandler.nBusy = 0; sqlite3_mutex_leave(db->mutex); assert( rc==SQLITE_OK || (*ppStmt)==0 ); return rc; } /* ** Rerun the compilation of a statement after a schema change. ** |
︙ | ︙ | |||
141396 141397 141398 141399 141400 141401 141402 141403 141404 141405 141406 141407 141408 141409 | /* ** Delete the given Select structure and all of its substructures. */ SQLITE_PRIVATE void sqlite3SelectDelete(sqlite3 *db, Select *p){ if( OK_IF_ALWAYS_TRUE(p) ) clearSelect(db, p, 1); } /* ** Return a pointer to the right-most SELECT statement in a compound. */ static Select *findRightmost(Select *p){ while( p->pNext ) p = p->pNext; | > > > | 141900 141901 141902 141903 141904 141905 141906 141907 141908 141909 141910 141911 141912 141913 141914 141915 141916 | /* ** Delete the given Select structure and all of its substructures. */ SQLITE_PRIVATE void sqlite3SelectDelete(sqlite3 *db, Select *p){ if( OK_IF_ALWAYS_TRUE(p) ) clearSelect(db, p, 1); } SQLITE_PRIVATE void sqlite3SelectDeleteGeneric(sqlite3 *db, void *p){ if( ALWAYS(p) ) clearSelect(db, (Select*)p, 1); } /* ** Return a pointer to the right-most SELECT statement in a compound. */ static Select *findRightmost(Select *p){ while( p->pNext ) p = p->pNext; |
︙ | ︙ | |||
143532 143533 143534 143535 143536 143537 143538 | CollSeq *pColl; int i,j; Expr *p; struct ExprList_item *a; NameContext sNC; assert( pSelect!=0 ); | > | | 144039 144040 144041 144042 144043 144044 144045 144046 144047 144048 144049 144050 144051 144052 144053 144054 | CollSeq *pColl; int i,j; Expr *p; struct ExprList_item *a; NameContext sNC; assert( pSelect!=0 ); testcase( (pSelect->selFlags & SF_Resolved)==0 ); assert( (pSelect->selFlags & SF_Resolved)!=0 || IN_RENAME_OBJECT ); assert( pTab->nCol==pSelect->pEList->nExpr || pParse->nErr>0 ); assert( aff==SQLITE_AFF_NONE || aff==SQLITE_AFF_BLOB ); if( db->mallocFailed || IN_RENAME_OBJECT ) return; while( pSelect->pPrior ) pSelect = pSelect->pPrior; a = pSelect->pEList->a; memset(&sNC, 0, sizeof(sNC)); sNC.pSrcList = pSelect->pSrc; |
︙ | ︙ | |||
144416 144417 144418 144419 144420 144421 144422 | sqlite3KeyInfoUnref(pKeyInfo); } multi_select_end: pDest->iSdst = dest.iSdst; pDest->nSdst = dest.nSdst; if( pDelete ){ | | < < | 144924 144925 144926 144927 144928 144929 144930 144931 144932 144933 144934 144935 144936 144937 144938 | sqlite3KeyInfoUnref(pKeyInfo); } multi_select_end: pDest->iSdst = dest.iSdst; pDest->nSdst = dest.nSdst; if( pDelete ){ sqlite3ParserAddCleanup(pParse, sqlite3SelectDeleteGeneric, pDelete); } return rc; } #endif /* SQLITE_OMIT_COMPOUND_SELECT */ /* ** Error message for when two or more terms of a compound select have different |
︙ | ︙ | |||
144969 144970 144971 144972 144973 144974 144975 | /* Jump to the this point in order to terminate the query. */ sqlite3VdbeResolveLabel(v, labelEnd); /* Make arrangements to free the 2nd and subsequent arms of the compound ** after the parse has finished */ if( pSplit->pPrior ){ | | < | 145475 145476 145477 145478 145479 145480 145481 145482 145483 145484 145485 145486 145487 145488 145489 | /* Jump to the this point in order to terminate the query. */ sqlite3VdbeResolveLabel(v, labelEnd); /* Make arrangements to free the 2nd and subsequent arms of the compound ** after the parse has finished */ if( pSplit->pPrior ){ sqlite3ParserAddCleanup(pParse, sqlite3SelectDeleteGeneric, pSplit->pPrior); } pSplit->pPrior = pPrior; pPrior->pNext = pSplit; sqlite3ExprListDelete(db, pPrior->pOrderBy); pPrior->pOrderBy = 0; /*** TBD: Insert subroutine calls to close cursors on incomplete |
︙ | ︙ | |||
145791 145792 145793 145794 145795 145796 145797 | ** ** pSubitem->pTab is always non-NULL by test restrictions and tests above. */ if( ALWAYS(pSubitem->pTab!=0) ){ Table *pTabToDel = pSubitem->pTab; if( pTabToDel->nTabRef==1 ){ Parse *pToplevel = sqlite3ParseToplevel(pParse); | | < < | 146296 146297 146298 146299 146300 146301 146302 146303 146304 146305 146306 146307 146308 146309 146310 | ** ** pSubitem->pTab is always non-NULL by test restrictions and tests above. */ if( ALWAYS(pSubitem->pTab!=0) ){ Table *pTabToDel = pSubitem->pTab; if( pTabToDel->nTabRef==1 ){ Parse *pToplevel = sqlite3ParseToplevel(pParse); sqlite3ParserAddCleanup(pToplevel, sqlite3DeleteTableGeneric, pTabToDel); testcase( pToplevel->earlyCleanup ); }else{ pTabToDel->nTabRef--; } pSubitem->pTab = 0; } |
︙ | ︙ | |||
146840 146841 146842 146843 146844 146845 146846 | ** ** If bFree is true, do not continue to use the pWith pointer after ** calling this routine, Instead, use only the return value. */ SQLITE_PRIVATE With *sqlite3WithPush(Parse *pParse, With *pWith, u8 bFree){ if( pWith ){ if( bFree ){ | | < | 147343 147344 147345 147346 147347 147348 147349 147350 147351 147352 147353 147354 147355 147356 147357 | ** ** If bFree is true, do not continue to use the pWith pointer after ** calling this routine, Instead, use only the return value. */ SQLITE_PRIVATE With *sqlite3WithPush(Parse *pParse, With *pWith, u8 bFree){ if( pWith ){ if( bFree ){ pWith = (With*)sqlite3ParserAddCleanup(pParse, sqlite3WithDeleteGeneric, pWith); if( pWith==0 ) return 0; } if( pParse->nErr==0 ){ assert( pParse->pWith!=pWith ); pWith->pOuter = pParse->pWith; pParse->pWith = pWith; |
︙ | ︙ | |||
147585 147586 147587 147588 147589 147590 147591 | */ static void selectAddSubqueryTypeInfo(Walker *pWalker, Select *p){ Parse *pParse; int i; SrcList *pTabList; SrcItem *pFrom; | < > > | 148087 148088 148089 148090 148091 148092 148093 148094 148095 148096 148097 148098 148099 148100 148101 148102 148103 148104 148105 | */ static void selectAddSubqueryTypeInfo(Walker *pWalker, Select *p){ Parse *pParse; int i; SrcList *pTabList; SrcItem *pFrom; if( p->selFlags & SF_HasTypeInfo ) return; p->selFlags |= SF_HasTypeInfo; pParse = pWalker->pParse; testcase( (p->selFlags & SF_Resolved)==0 ); assert( (p->selFlags & SF_Resolved) || IN_RENAME_OBJECT ); pTabList = p->pSrc; for(i=0, pFrom=pTabList->a; i<pTabList->nSrc; i++, pFrom++){ Table *pTab = pFrom->pTab; assert( pTab!=0 ); if( (pTab->tabFlags & TF_Ephemeral)!=0 ){ /* A sub-query in the FROM clause of a SELECT */ Select *pSel = pFrom->pSelect; |
︙ | ︙ | |||
147873 147874 147875 147876 147877 147878 147879 147880 147881 147882 147883 147884 147885 147886 147887 147888 147889 147890 147891 147892 147893 147894 147895 | if( pFunc->iOBTab>=0 ){ ExprList *pOBList; KeyInfo *pKeyInfo; int nExtra = 0; assert( pFunc->pFExpr->pLeft!=0 ); assert( pFunc->pFExpr->pLeft->op==TK_ORDER ); assert( ExprUseXList(pFunc->pFExpr->pLeft) ); pOBList = pFunc->pFExpr->pLeft->x.pList; if( !pFunc->bOBUnique ){ nExtra++; /* One extra column for the OP_Sequence */ } if( pFunc->bOBPayload ){ /* extra columns for the function arguments */ assert( ExprUseXList(pFunc->pFExpr) ); nExtra += pFunc->pFExpr->x.pList->nExpr; } pKeyInfo = sqlite3KeyInfoFromExprList(pParse, pOBList, 0, nExtra); if( !pFunc->bOBUnique && pParse->nErr==0 ){ pKeyInfo->nKeyField++; } sqlite3VdbeAddOp4(v, OP_OpenEphemeral, pFunc->iOBTab, pOBList->nExpr+nExtra, 0, (char*)pKeyInfo, P4_KEYINFO); | > > > > | 148376 148377 148378 148379 148380 148381 148382 148383 148384 148385 148386 148387 148388 148389 148390 148391 148392 148393 148394 148395 148396 148397 148398 148399 148400 148401 148402 | if( pFunc->iOBTab>=0 ){ ExprList *pOBList; KeyInfo *pKeyInfo; int nExtra = 0; assert( pFunc->pFExpr->pLeft!=0 ); assert( pFunc->pFExpr->pLeft->op==TK_ORDER ); assert( ExprUseXList(pFunc->pFExpr->pLeft) ); assert( pFunc->pFunc!=0 ); pOBList = pFunc->pFExpr->pLeft->x.pList; if( !pFunc->bOBUnique ){ nExtra++; /* One extra column for the OP_Sequence */ } if( pFunc->bOBPayload ){ /* extra columns for the function arguments */ assert( ExprUseXList(pFunc->pFExpr) ); nExtra += pFunc->pFExpr->x.pList->nExpr; } if( pFunc->bUseSubtype ){ nExtra += pFunc->pFExpr->x.pList->nExpr; } pKeyInfo = sqlite3KeyInfoFromExprList(pParse, pOBList, 0, nExtra); if( !pFunc->bOBUnique && pParse->nErr==0 ){ pKeyInfo->nKeyField++; } sqlite3VdbeAddOp4(v, OP_OpenEphemeral, pFunc->iOBTab, pOBList->nExpr+nExtra, 0, (char*)pKeyInfo, P4_KEYINFO); |
︙ | ︙ | |||
147908 147909 147910 147911 147912 147913 147914 | int i; struct AggInfo_func *pF; for(i=0, pF=pAggInfo->aFunc; i<pAggInfo->nFunc; i++, pF++){ ExprList *pList; assert( ExprUseXList(pF->pFExpr) ); pList = pF->pFExpr->x.pList; if( pF->iOBTab>=0 ){ | | | | > | > > > > > > > > > | 148415 148416 148417 148418 148419 148420 148421 148422 148423 148424 148425 148426 148427 148428 148429 148430 148431 148432 148433 148434 148435 148436 148437 148438 148439 148440 148441 148442 148443 148444 148445 148446 148447 148448 148449 148450 148451 148452 148453 148454 148455 148456 148457 148458 148459 148460 148461 148462 148463 148464 | int i; struct AggInfo_func *pF; for(i=0, pF=pAggInfo->aFunc; i<pAggInfo->nFunc; i++, pF++){ ExprList *pList; assert( ExprUseXList(pF->pFExpr) ); pList = pF->pFExpr->x.pList; if( pF->iOBTab>=0 ){ /* For an ORDER BY aggregate, calls to OP_AggStep were deferred. Inputs ** were stored in emphermal table pF->iOBTab. Here, we extract those ** inputs (in ORDER BY order) and make all calls to OP_AggStep ** before doing the OP_AggFinal call. */ int iTop; /* Start of loop for extracting columns */ int nArg; /* Number of columns to extract */ int nKey; /* Key columns to be skipped */ int regAgg; /* Extract into this array */ int j; /* Loop counter */ assert( pF->pFunc!=0 ); nArg = pList->nExpr; regAgg = sqlite3GetTempRange(pParse, nArg); if( pF->bOBPayload==0 ){ nKey = 0; }else{ assert( pF->pFExpr->pLeft!=0 ); assert( ExprUseXList(pF->pFExpr->pLeft) ); assert( pF->pFExpr->pLeft->x.pList!=0 ); nKey = pF->pFExpr->pLeft->x.pList->nExpr; if( ALWAYS(!pF->bOBUnique) ) nKey++; } iTop = sqlite3VdbeAddOp1(v, OP_Rewind, pF->iOBTab); VdbeCoverage(v); for(j=nArg-1; j>=0; j--){ sqlite3VdbeAddOp3(v, OP_Column, pF->iOBTab, nKey+j, regAgg+j); } if( pF->bUseSubtype ){ int regSubtype = sqlite3GetTempReg(pParse); int iBaseCol = nKey + nArg + (pF->bOBPayload==0 && pF->bOBUnique==0); for(j=nArg-1; j>=0; j--){ sqlite3VdbeAddOp3(v, OP_Column, pF->iOBTab, iBaseCol+j, regSubtype); sqlite3VdbeAddOp2(v, OP_SetSubtype, regSubtype, regAgg+j); } sqlite3ReleaseTempReg(pParse, regSubtype); } sqlite3VdbeAddOp3(v, OP_AggStep, 0, regAgg, AggInfoFuncReg(pAggInfo,i)); sqlite3VdbeAppendP4(v, pF->pFunc, P4_FUNCDEF); sqlite3VdbeChangeP5(v, (u8)nArg); sqlite3VdbeAddOp2(v, OP_Next, pF->iOBTab, iTop+1); VdbeCoverage(v); sqlite3VdbeJumpHere(v, iTop); sqlite3ReleaseTempRange(pParse, regAgg, nArg); } |
︙ | ︙ | |||
147956 147957 147958 147959 147960 147961 147962 | ** aggregate based on the current cursor position. ** ** If regAcc is non-zero and there are no min() or max() aggregates ** in pAggInfo, then only populate the pAggInfo->nAccumulator accumulator ** registers if register regAcc contains 0. The caller will take care ** of setting and clearing regAcc. ** | | | | 148473 148474 148475 148476 148477 148478 148479 148480 148481 148482 148483 148484 148485 148486 148487 148488 148489 148490 | ** aggregate based on the current cursor position. ** ** If regAcc is non-zero and there are no min() or max() aggregates ** in pAggInfo, then only populate the pAggInfo->nAccumulator accumulator ** registers if register regAcc contains 0. The caller will take care ** of setting and clearing regAcc. ** ** For an ORDER BY aggregate, the actual accumulator memory cell update ** is deferred until after all input rows have been received, so that they ** can be run in the requested order. In that case, instead of invoking ** OP_AggStep to update the accumulator, just add the arguments that would ** have been passed into OP_AggStep into the sorting ephemeral table ** (along with the appropriate sort key). */ static void updateAccumulator( Parse *pParse, int regAcc, AggInfo *pAggInfo, |
︙ | ︙ | |||
147984 147985 147986 147987 147988 147989 147990 147991 147992 147993 147994 147995 147996 147997 147998 147999 148000 | if( pParse->nErr ) return; pAggInfo->directMode = 1; for(i=0, pF=pAggInfo->aFunc; i<pAggInfo->nFunc; i++, pF++){ int nArg; int addrNext = 0; int regAgg; int regAggSz = 0; ExprList *pList; assert( ExprUseXList(pF->pFExpr) ); assert( !IsWindowFunc(pF->pFExpr) ); pList = pF->pFExpr->x.pList; if( ExprHasProperty(pF->pFExpr, EP_WinFunc) ){ Expr *pFilter = pF->pFExpr->y.pWin->pFilter; if( pAggInfo->nAccumulator && (pF->pFunc->funcFlags & SQLITE_FUNC_NEEDCOLL) && regAcc ){ | > > | 148501 148502 148503 148504 148505 148506 148507 148508 148509 148510 148511 148512 148513 148514 148515 148516 148517 148518 148519 | if( pParse->nErr ) return; pAggInfo->directMode = 1; for(i=0, pF=pAggInfo->aFunc; i<pAggInfo->nFunc; i++, pF++){ int nArg; int addrNext = 0; int regAgg; int regAggSz = 0; int regDistinct = 0; ExprList *pList; assert( ExprUseXList(pF->pFExpr) ); assert( !IsWindowFunc(pF->pFExpr) ); assert( pF->pFunc!=0 ); pList = pF->pFExpr->x.pList; if( ExprHasProperty(pF->pFExpr, EP_WinFunc) ){ Expr *pFilter = pF->pFExpr->y.pWin->pFilter; if( pAggInfo->nAccumulator && (pF->pFunc->funcFlags & SQLITE_FUNC_NEEDCOLL) && regAcc ){ |
︙ | ︙ | |||
148030 148031 148032 148033 148034 148035 148036 148037 148038 148039 148040 148041 148042 148043 148044 148045 148046 | assert( pOBList->nExpr>0 ); regAggSz = pOBList->nExpr; if( !pF->bOBUnique ){ regAggSz++; /* One register for OP_Sequence */ } if( pF->bOBPayload ){ regAggSz += nArg; } regAggSz++; /* One extra register to hold result of MakeRecord */ regAgg = sqlite3GetTempRange(pParse, regAggSz); sqlite3ExprCodeExprList(pParse, pOBList, regAgg, 0, SQLITE_ECEL_DUP); jj = pOBList->nExpr; if( !pF->bOBUnique ){ sqlite3VdbeAddOp2(v, OP_Sequence, pF->iOBTab, regAgg+jj); jj++; } if( pF->bOBPayload ){ | > > > > > | > > > > > > > > > | | 148549 148550 148551 148552 148553 148554 148555 148556 148557 148558 148559 148560 148561 148562 148563 148564 148565 148566 148567 148568 148569 148570 148571 148572 148573 148574 148575 148576 148577 148578 148579 148580 148581 148582 148583 148584 148585 148586 148587 148588 148589 148590 148591 148592 148593 148594 148595 148596 148597 148598 148599 148600 148601 148602 | assert( pOBList->nExpr>0 ); regAggSz = pOBList->nExpr; if( !pF->bOBUnique ){ regAggSz++; /* One register for OP_Sequence */ } if( pF->bOBPayload ){ regAggSz += nArg; } if( pF->bUseSubtype ){ regAggSz += nArg; } regAggSz++; /* One extra register to hold result of MakeRecord */ regAgg = sqlite3GetTempRange(pParse, regAggSz); regDistinct = regAgg; sqlite3ExprCodeExprList(pParse, pOBList, regAgg, 0, SQLITE_ECEL_DUP); jj = pOBList->nExpr; if( !pF->bOBUnique ){ sqlite3VdbeAddOp2(v, OP_Sequence, pF->iOBTab, regAgg+jj); jj++; } if( pF->bOBPayload ){ regDistinct = regAgg+jj; sqlite3ExprCodeExprList(pParse, pList, regDistinct, 0, SQLITE_ECEL_DUP); jj += nArg; } if( pF->bUseSubtype ){ int kk; int regBase = pF->bOBPayload ? regDistinct : regAgg; for(kk=0; kk<nArg; kk++, jj++){ sqlite3VdbeAddOp2(v, OP_GetSubtype, regBase+kk, regAgg+jj); } } }else if( pList ){ nArg = pList->nExpr; regAgg = sqlite3GetTempRange(pParse, nArg); regDistinct = regAgg; sqlite3ExprCodeExprList(pParse, pList, regAgg, 0, SQLITE_ECEL_DUP); }else{ nArg = 0; regAgg = 0; } if( pF->iDistinct>=0 && pList ){ if( addrNext==0 ){ addrNext = sqlite3VdbeMakeLabel(pParse); } pF->iDistinct = codeDistinct(pParse, eDistinctType, pF->iDistinct, addrNext, pList, regDistinct); } if( pF->iOBTab>=0 ){ /* Insert a new record into the ORDER BY table */ sqlite3VdbeAddOp3(v, OP_MakeRecord, regAgg, regAggSz-1, regAgg+regAggSz-1); sqlite3VdbeAddOp4Int(v, OP_IdxInsert, pF->iOBTab, regAgg+regAggSz-1, regAgg, regAggSz-1); |
︙ | ︙ | |||
148244 148245 148246 148247 148248 148249 148250 | } return 0; } /* ** Deallocate a single AggInfo object */ | | > | 148777 148778 148779 148780 148781 148782 148783 148784 148785 148786 148787 148788 148789 148790 148791 148792 | } return 0; } /* ** Deallocate a single AggInfo object */ static void agginfoFree(sqlite3 *db, void *pArg){ AggInfo *p = (AggInfo*)pArg; sqlite3DbFree(db, p->aCol); sqlite3DbFree(db, p->aFunc); sqlite3DbFreeNN(db, p); } /* ** Attempt to transform a query of the form |
︙ | ︙ | |||
148318 148319 148320 148321 148322 148323 148324 | Expr *pTerm; pPrior = pSub->pPrior; pSub->pPrior = 0; pSub->pNext = 0; pSub->selFlags |= SF_Aggregate; pSub->selFlags &= ~SF_Compound; pSub->nSelectRow = 0; | | | 148852 148853 148854 148855 148856 148857 148858 148859 148860 148861 148862 148863 148864 148865 148866 | Expr *pTerm; pPrior = pSub->pPrior; pSub->pPrior = 0; pSub->pNext = 0; pSub->selFlags |= SF_Aggregate; pSub->selFlags &= ~SF_Compound; pSub->nSelectRow = 0; sqlite3ParserAddCleanup(pParse, sqlite3ExprListDeleteGeneric, pSub->pEList); pTerm = pPrior ? sqlite3ExprDup(db, pCount, 0) : pCount; pSub->pEList = sqlite3ExprListAppend(pParse, 0, pTerm); pTerm = sqlite3PExpr(pParse, TK_SELECT, 0, 0); sqlite3PExprAddSelect(pParse, pTerm, pSub); if( pExpr==0 ){ pExpr = pTerm; }else{ |
︙ | ︙ | |||
148498 148499 148500 148501 148502 148503 148504 | if( p->pOrderBy ){ #if TREETRACE_ENABLED TREETRACE(0x800,pParse,p, ("dropping superfluous ORDER BY:\n")); if( sqlite3TreeTrace & 0x800 ){ sqlite3TreeViewExprList(0, p->pOrderBy, 0, "ORDERBY"); } #endif | | < | | 149032 149033 149034 149035 149036 149037 149038 149039 149040 149041 149042 149043 149044 149045 149046 149047 | if( p->pOrderBy ){ #if TREETRACE_ENABLED TREETRACE(0x800,pParse,p, ("dropping superfluous ORDER BY:\n")); if( sqlite3TreeTrace & 0x800 ){ sqlite3TreeViewExprList(0, p->pOrderBy, 0, "ORDERBY"); } #endif sqlite3ParserAddCleanup(pParse, sqlite3ExprListDeleteGeneric, p->pOrderBy); testcase( pParse->earlyCleanup ); p->pOrderBy = 0; } p->selFlags &= ~SF_Distinct; p->selFlags |= SF_NoopOrderBy; } sqlite3SelectPrep(pParse, p, 0); |
︙ | ︙ | |||
148606 148607 148608 148609 148610 148611 148612 148613 148614 148615 148616 148617 148618 148619 148620 148621 148622 148623 148624 148625 148626 148627 148628 148629 | TREETRACE(0x1000,pParse,p, ("FULL-JOIN simplifies to RIGHT-JOIN on term %d\n",i)); pItem->fg.jointype &= ~JT_LEFT; }else{ TREETRACE(0x1000,pParse,p, ("LEFT-JOIN simplifies to JOIN on term %d\n",i)); pItem->fg.jointype &= ~(JT_LEFT|JT_OUTER); } } if( pItem->fg.jointype & JT_LTORJ ){ for(j=i+1; j<pTabList->nSrc; j++){ SrcItem *pI2 = &pTabList->a[j]; if( pI2->fg.jointype & JT_RIGHT ){ if( pI2->fg.jointype & JT_LEFT ){ TREETRACE(0x1000,pParse,p, ("FULL-JOIN simplifies to LEFT-JOIN on term %d\n",j)); pI2->fg.jointype &= ~JT_RIGHT; }else{ TREETRACE(0x1000,pParse,p, ("RIGHT-JOIN simplifies to JOIN on term %d\n",j)); pI2->fg.jointype &= ~(JT_RIGHT|JT_OUTER); } } } | > > | < < < | 149139 149140 149141 149142 149143 149144 149145 149146 149147 149148 149149 149150 149151 149152 149153 149154 149155 149156 149157 149158 149159 149160 149161 149162 149163 149164 149165 149166 149167 149168 149169 149170 149171 149172 149173 149174 149175 149176 | TREETRACE(0x1000,pParse,p, ("FULL-JOIN simplifies to RIGHT-JOIN on term %d\n",i)); pItem->fg.jointype &= ~JT_LEFT; }else{ TREETRACE(0x1000,pParse,p, ("LEFT-JOIN simplifies to JOIN on term %d\n",i)); pItem->fg.jointype &= ~(JT_LEFT|JT_OUTER); unsetJoinExpr(p->pWhere, pItem->iCursor, 0); } } if( pItem->fg.jointype & JT_LTORJ ){ for(j=i+1; j<pTabList->nSrc; j++){ SrcItem *pI2 = &pTabList->a[j]; if( pI2->fg.jointype & JT_RIGHT ){ if( pI2->fg.jointype & JT_LEFT ){ TREETRACE(0x1000,pParse,p, ("FULL-JOIN simplifies to LEFT-JOIN on term %d\n",j)); pI2->fg.jointype &= ~JT_RIGHT; }else{ TREETRACE(0x1000,pParse,p, ("RIGHT-JOIN simplifies to JOIN on term %d\n",j)); pI2->fg.jointype &= ~(JT_RIGHT|JT_OUTER); unsetJoinExpr(p->pWhere, pI2->iCursor, 1); } } } for(j=pTabList->nSrc-1; j>=0; j--){ pTabList->a[j].fg.jointype &= ~JT_LTORJ; if( pTabList->a[j].fg.jointype & JT_RIGHT ) break; } } } /* No further action if this term of the FROM clause is not a subquery */ if( pSub==0 ) continue; /* Catch mismatch in the declared columns of a view and the number of ** columns in the SELECT on the RHS */ |
︙ | ︙ | |||
148693 148694 148695 148696 148697 148698 148699 | && pSub->pLimit==0 /* Condition (1) */ && (pSub->selFlags & SF_OrderByReqd)==0 /* Condition (2) */ && (p->selFlags & SF_OrderByReqd)==0 /* Condition (3) and (4) */ && OptimizationEnabled(db, SQLITE_OmitOrderBy) ){ TREETRACE(0x800,pParse,p, ("omit superfluous ORDER BY on %r FROM-clause subquery\n",i+1)); | | < | | 149225 149226 149227 149228 149229 149230 149231 149232 149233 149234 149235 149236 149237 149238 149239 149240 | && pSub->pLimit==0 /* Condition (1) */ && (pSub->selFlags & SF_OrderByReqd)==0 /* Condition (2) */ && (p->selFlags & SF_OrderByReqd)==0 /* Condition (3) and (4) */ && OptimizationEnabled(db, SQLITE_OmitOrderBy) ){ TREETRACE(0x800,pParse,p, ("omit superfluous ORDER BY on %r FROM-clause subquery\n",i+1)); sqlite3ParserAddCleanup(pParse, sqlite3ExprListDeleteGeneric, pSub->pOrderBy); pSub->pOrderBy = 0; } /* If the outer query contains a "complex" result set (that is, ** if the result set of the outer query uses functions or subqueries) ** and if the subquery contains an ORDER BY clause and if ** it will be implemented as a co-routine, then do not flatten. This |
︙ | ︙ | |||
149224 149225 149226 149227 149228 149229 149230 | /* Convert TK_COLUMN nodes into TK_AGG_COLUMN and make entries in ** sAggInfo for all TK_AGG_FUNCTION nodes in expressions of the ** SELECT statement. */ pAggInfo = sqlite3DbMallocZero(db, sizeof(*pAggInfo) ); if( pAggInfo ){ | | < | 149755 149756 149757 149758 149759 149760 149761 149762 149763 149764 149765 149766 149767 149768 149769 | /* Convert TK_COLUMN nodes into TK_AGG_COLUMN and make entries in ** sAggInfo for all TK_AGG_FUNCTION nodes in expressions of the ** SELECT statement. */ pAggInfo = sqlite3DbMallocZero(db, sizeof(*pAggInfo) ); if( pAggInfo ){ sqlite3ParserAddCleanup(pParse, agginfoFree, pAggInfo); testcase( pParse->earlyCleanup ); } if( db->mallocFailed ){ goto select_end; } pAggInfo->selId = p->selId; #ifdef SQLITE_DEBUG |
︙ | ︙ | |||
150933 150934 150935 150936 150937 150938 150939 | sqlite3 *db = pParse->db; ExprList *pNew; Returning *pReturning; Select sSelect; SrcList sFrom; assert( v!=0 ); | | > > > > | > > > | 151463 151464 151465 151466 151467 151468 151469 151470 151471 151472 151473 151474 151475 151476 151477 151478 151479 151480 151481 151482 151483 151484 151485 151486 151487 | sqlite3 *db = pParse->db; ExprList *pNew; Returning *pReturning; Select sSelect; SrcList sFrom; assert( v!=0 ); if( !pParse->bReturning ){ /* This RETURNING trigger must be for a different statement as ** this statement lacks a RETURNING clause. */ return; } assert( db->pParse==pParse ); pReturning = pParse->u1.pReturning; if( pTrigger != &(pReturning->retTrig) ){ /* This RETURNING trigger is for a different statement */ return; } memset(&sSelect, 0, sizeof(sSelect)); memset(&sFrom, 0, sizeof(sFrom)); sSelect.pEList = sqlite3ExprListDup(db, pReturning->pReturnEL, 0); sSelect.pSrc = &sFrom; sFrom.nSrc = 1; sFrom.a[0].pTab = pTab; sFrom.a[0].iCursor = -1; |
︙ | ︙ | |||
153867 153868 153869 153870 153871 153872 153873 | VTable *p = db->pDisconnect; assert( sqlite3BtreeHoldsAllMutexes(db) ); assert( sqlite3_mutex_held(db->mutex) ); if( p ){ db->pDisconnect = 0; | < | 154404 154405 154406 154407 154408 154409 154410 154411 154412 154413 154414 154415 154416 154417 | VTable *p = db->pDisconnect; assert( sqlite3BtreeHoldsAllMutexes(db) ); assert( sqlite3_mutex_held(db->mutex) ); if( p ){ db->pDisconnect = 0; do { VTable *pNext = p->pNext; sqlite3VtabUnlock(p); p = pNext; }while( p ); } } |
︙ | ︙ | |||
155433 155434 155435 155436 155437 155438 155439 | ** ** where.c: */ SQLITE_PRIVATE Bitmask sqlite3WhereGetMask(WhereMaskSet*,int); #ifdef WHERETRACE_ENABLED SQLITE_PRIVATE void sqlite3WhereClausePrint(WhereClause *pWC); SQLITE_PRIVATE void sqlite3WhereTermPrint(WhereTerm *pTerm, int iTerm); | | | 155969 155970 155971 155972 155973 155974 155975 155976 155977 155978 155979 155980 155981 155982 155983 | ** ** where.c: */ SQLITE_PRIVATE Bitmask sqlite3WhereGetMask(WhereMaskSet*,int); #ifdef WHERETRACE_ENABLED SQLITE_PRIVATE void sqlite3WhereClausePrint(WhereClause *pWC); SQLITE_PRIVATE void sqlite3WhereTermPrint(WhereTerm *pTerm, int iTerm); SQLITE_PRIVATE void sqlite3WhereLoopPrint(const WhereLoop *p, const WhereClause *pWC); #endif SQLITE_PRIVATE WhereTerm *sqlite3WhereFindTerm( WhereClause *pWC, /* The WHERE clause to be searched */ int iCur, /* Cursor number of LHS */ int iColumn, /* Column number of LHS */ Bitmask notReady, /* RHS must not overlap with this mask */ u32 op, /* Mask of WO_xx values describing operator */ |
︙ | ︙ | |||
160895 160896 160897 160898 160899 160900 160901 160902 160903 160904 160905 160906 160907 160908 160909 160910 160911 160912 160913 160914 | Vdbe *v = pParse->pVdbe; VdbeOp *pOp = sqlite3VdbeGetOp(v, iStart); int iEnd = sqlite3VdbeCurrentAddr(v); if( pParse->db->mallocFailed ) return; for(; iStart<iEnd; iStart++, pOp++){ if( pOp->p1!=iTabCur ) continue; if( pOp->opcode==OP_Column ){ pOp->opcode = OP_Copy; pOp->p1 = pOp->p2 + iRegister; pOp->p2 = pOp->p3; pOp->p3 = 0; pOp->p5 = 2; /* Cause the MEM_Subtype flag to be cleared */ }else if( pOp->opcode==OP_Rowid ){ pOp->opcode = OP_Sequence; pOp->p1 = iAutoidxCur; #ifdef SQLITE_ALLOW_ROWID_IN_VIEW if( iAutoidxCur==0 ){ pOp->opcode = OP_Null; pOp->p3 = 0; } | > > > > > > > > > > | 161431 161432 161433 161434 161435 161436 161437 161438 161439 161440 161441 161442 161443 161444 161445 161446 161447 161448 161449 161450 161451 161452 161453 161454 161455 161456 161457 161458 161459 161460 | Vdbe *v = pParse->pVdbe; VdbeOp *pOp = sqlite3VdbeGetOp(v, iStart); int iEnd = sqlite3VdbeCurrentAddr(v); if( pParse->db->mallocFailed ) return; for(; iStart<iEnd; iStart++, pOp++){ if( pOp->p1!=iTabCur ) continue; if( pOp->opcode==OP_Column ){ #ifdef SQLITE_DEBUG if( pParse->db->flags & SQLITE_VdbeAddopTrace ){ printf("TRANSLATE OP_Column to OP_Copy at %d\n", iStart); } #endif pOp->opcode = OP_Copy; pOp->p1 = pOp->p2 + iRegister; pOp->p2 = pOp->p3; pOp->p3 = 0; pOp->p5 = 2; /* Cause the MEM_Subtype flag to be cleared */ }else if( pOp->opcode==OP_Rowid ){ #ifdef SQLITE_DEBUG if( pParse->db->flags & SQLITE_VdbeAddopTrace ){ printf("TRANSLATE OP_Rowid to OP_Sequence at %d\n", iStart); } #endif pOp->opcode = OP_Sequence; pOp->p1 = iAutoidxCur; #ifdef SQLITE_ALLOW_ROWID_IN_VIEW if( iAutoidxCur==0 ){ pOp->opcode = OP_Null; pOp->p3 = 0; } |
︙ | ︙ | |||
162227 162228 162229 162230 162231 162232 162233 | if( rc==SQLITE_OK ){ if( iUpper>iLower ){ nNew = sqlite3LogEst(iUpper - iLower); /* TUNING: If both iUpper and iLower are derived from the same ** sample, then assume they are 4x more selective. This brings ** the estimated selectivity more in line with what it would be ** if estimated without the use of STAT4 tables. */ | | > | 162773 162774 162775 162776 162777 162778 162779 162780 162781 162782 162783 162784 162785 162786 162787 162788 | if( rc==SQLITE_OK ){ if( iUpper>iLower ){ nNew = sqlite3LogEst(iUpper - iLower); /* TUNING: If both iUpper and iLower are derived from the same ** sample, then assume they are 4x more selective. This brings ** the estimated selectivity more in line with what it would be ** if estimated without the use of STAT4 tables. */ if( iLwrIdx==iUprIdx ){ nNew -= 20; } assert( 20==sqlite3LogEst(4) ); }else{ nNew = 10; assert( 10==sqlite3LogEst(2) ); } if( nNew<nOut ){ nOut = nNew; } WHERETRACE(0x20, ("STAT4 range scan: %u..%u est=%d\n", |
︙ | ︙ | |||
162451 162452 162453 162454 162455 162456 162457 162458 | } } #endif #ifdef WHERETRACE_ENABLED /* ** Print a WhereLoop object for debugging purposes */ | > > > > > > > > > > > > | > | | | | | | | | | > > > > | 162998 162999 163000 163001 163002 163003 163004 163005 163006 163007 163008 163009 163010 163011 163012 163013 163014 163015 163016 163017 163018 163019 163020 163021 163022 163023 163024 163025 163026 163027 163028 163029 163030 163031 163032 163033 163034 163035 163036 163037 163038 163039 | } } #endif #ifdef WHERETRACE_ENABLED /* ** Print a WhereLoop object for debugging purposes ** ** Format example: ** ** .--- Position in WHERE clause rSetup, rRun, nOut ---. ** | | ** | .--- selfMask nTerm ------. | ** | | | | ** | | .-- prereq Idx wsFlags----. | | ** | | | Name | | | ** | | | __|__ nEq ---. ___|__ | __|__ ** | / \ / \ / \ | / \ / \ / \ ** 1.002.001 t2.t2xy 2 f 010241 N 2 cost 0,56,31 */ SQLITE_PRIVATE void sqlite3WhereLoopPrint(const WhereLoop *p, const WhereClause *pWC){ if( pWC ){ WhereInfo *pWInfo = pWC->pWInfo; int nb = 1+(pWInfo->pTabList->nSrc+3)/4; SrcItem *pItem = pWInfo->pTabList->a + p->iTab; Table *pTab = pItem->pTab; Bitmask mAll = (((Bitmask)1)<<(nb*4)) - 1; sqlite3DebugPrintf("%c%2d.%0*llx.%0*llx", p->cId, p->iTab, nb, p->maskSelf, nb, p->prereq & mAll); sqlite3DebugPrintf(" %12s", pItem->zAlias ? pItem->zAlias : pTab->zName); }else{ sqlite3DebugPrintf("%c%2d.%03llx.%03llx %c%d", p->cId, p->iTab, p->maskSelf, p->prereq & 0xfff, p->cId, p->iTab); } if( (p->wsFlags & WHERE_VIRTUALTABLE)==0 ){ const char *zName; if( p->u.btree.pIndex && (zName = p->u.btree.pIndex->zName)!=0 ){ if( strncmp(zName, "sqlite_autoindex_", 17)==0 ){ int i = sqlite3Strlen30(zName) - 1; while( zName[i]!='_' ) i--; zName += i; |
︙ | ︙ | |||
162497 162498 162499 162500 162501 162502 162503 162504 162505 162506 162507 162508 162509 162510 | sqlite3DebugPrintf(" cost %d,%d,%d\n", p->rSetup, p->rRun, p->nOut); if( p->nLTerm && (sqlite3WhereTrace & 0x4000)!=0 ){ int i; for(i=0; i<p->nLTerm; i++){ sqlite3WhereTermPrint(p->aLTerm[i], i); } } } #endif /* ** Convert bulk memory into a valid WhereLoop that can be passed ** to whereLoopClear harmlessly. */ | > > > > > > > > > | 163061 163062 163063 163064 163065 163066 163067 163068 163069 163070 163071 163072 163073 163074 163075 163076 163077 163078 163079 163080 163081 163082 163083 | sqlite3DebugPrintf(" cost %d,%d,%d\n", p->rSetup, p->rRun, p->nOut); if( p->nLTerm && (sqlite3WhereTrace & 0x4000)!=0 ){ int i; for(i=0; i<p->nLTerm; i++){ sqlite3WhereTermPrint(p->aLTerm[i], i); } } } SQLITE_PRIVATE void sqlite3ShowWhereLoop(const WhereLoop *p){ if( p ) sqlite3WhereLoopPrint(p, 0); } SQLITE_PRIVATE void sqlite3ShowWhereLoopList(const WhereLoop *p){ while( p ){ sqlite3ShowWhereLoop(p); p = p->pNextLoop; } } #endif /* ** Convert bulk memory into a valid WhereLoop that can be passed ** to whereLoopClear harmlessly. */ |
︙ | ︙ | |||
162610 162611 162612 162613 162614 162615 162616 | sqlite3DbNNFreeNN(db, pWInfo->pMemToFree); pWInfo->pMemToFree = pNext; } sqlite3DbNNFreeNN(db, pWInfo); } /* | | > > > > > > > > > > > > > > > | | | | | | < < < < < < < < < > > > > > > > > > | < | | | | | 163183 163184 163185 163186 163187 163188 163189 163190 163191 163192 163193 163194 163195 163196 163197 163198 163199 163200 163201 163202 163203 163204 163205 163206 163207 163208 163209 163210 163211 163212 163213 163214 163215 163216 163217 163218 163219 163220 163221 163222 163223 163224 163225 163226 163227 163228 163229 163230 163231 163232 163233 163234 163235 163236 163237 163238 163239 163240 163241 163242 163243 163244 163245 163246 163247 163248 163249 163250 | sqlite3DbNNFreeNN(db, pWInfo->pMemToFree); pWInfo->pMemToFree = pNext; } sqlite3DbNNFreeNN(db, pWInfo); } /* ** Return TRUE if X is a proper subset of Y but is of equal or less cost. ** In other words, return true if all constraints of X are also part of Y ** and Y has additional constraints that might speed the search that X lacks ** but the cost of running X is not more than the cost of running Y. ** ** In other words, return true if the cost relationwship between X and Y ** is inverted and needs to be adjusted. ** ** Case 1: ** ** (1a) X and Y use the same index. ** (1b) X has fewer == terms than Y ** (1c) Neither X nor Y use skip-scan ** (1d) X does not have a a greater cost than Y ** ** Case 2: ** ** (2a) X has the same or lower cost, or returns the same or fewer rows, ** than Y. ** (2b) X uses fewer WHERE clause terms than Y ** (2c) Every WHERE clause term used by X is also used by Y ** (2d) X skips at least as many columns as Y ** (2e) If X is a covering index, than Y is too */ static int whereLoopCheaperProperSubset( const WhereLoop *pX, /* First WhereLoop to compare */ const WhereLoop *pY /* Compare against this WhereLoop */ ){ int i, j; if( pX->rRun>pY->rRun && pX->nOut>pY->nOut ) return 0; /* (1d) and (2a) */ assert( (pX->wsFlags & WHERE_VIRTUALTABLE)==0 ); assert( (pY->wsFlags & WHERE_VIRTUALTABLE)==0 ); if( pX->u.btree.nEq < pY->u.btree.nEq /* (1b) */ && pX->u.btree.pIndex==pY->u.btree.pIndex /* (1a) */ && pX->nSkip==0 && pY->nSkip==0 /* (1c) */ ){ return 1; /* Case 1 is true */ } if( pX->nLTerm-pX->nSkip >= pY->nLTerm-pY->nSkip ){ return 0; /* (2b) */ } if( pY->nSkip > pX->nSkip ) return 0; /* (2d) */ for(i=pX->nLTerm-1; i>=0; i--){ if( pX->aLTerm[i]==0 ) continue; for(j=pY->nLTerm-1; j>=0; j--){ if( pY->aLTerm[j]==pX->aLTerm[i] ) break; } if( j<0 ) return 0; /* (2c) */ } if( (pX->wsFlags&WHERE_IDX_ONLY)!=0 && (pY->wsFlags&WHERE_IDX_ONLY)==0 ){ return 0; /* (2e) */ } return 1; /* Case 2 is true */ } /* ** Try to adjust the cost and number of output rows of WhereLoop pTemplate ** upwards or downwards so that: ** ** (1) pTemplate costs less than any other WhereLoops that are a proper |
︙ | ︙ | |||
163139 163140 163141 163142 163143 163144 163145 | assert( (pNew->wsFlags & WHERE_TOP_LIMIT)==0 ); if( pNew->wsFlags & WHERE_BTM_LIMIT ){ opMask = WO_LT|WO_LE; }else{ assert( pNew->u.btree.nBtm==0 ); opMask = WO_EQ|WO_IN|WO_GT|WO_GE|WO_LT|WO_LE|WO_ISNULL|WO_IS; } | > | > > | 163726 163727 163728 163729 163730 163731 163732 163733 163734 163735 163736 163737 163738 163739 163740 163741 163742 163743 | assert( (pNew->wsFlags & WHERE_TOP_LIMIT)==0 ); if( pNew->wsFlags & WHERE_BTM_LIMIT ){ opMask = WO_LT|WO_LE; }else{ assert( pNew->u.btree.nBtm==0 ); opMask = WO_EQ|WO_IN|WO_GT|WO_GE|WO_LT|WO_LE|WO_ISNULL|WO_IS; } if( pProbe->bUnordered || pProbe->bLowQual ){ if( pProbe->bUnordered ) opMask &= ~(WO_GT|WO_GE|WO_LT|WO_LE); if( pProbe->bLowQual ) opMask &= ~(WO_EQ|WO_IN|WO_IS); } assert( pNew->u.btree.nEq<pProbe->nColumn ); assert( pNew->u.btree.nEq<pProbe->nKeyCol || pProbe->idxType!=SQLITE_IDXTYPE_PRIMARYKEY ); saved_nEq = pNew->u.btree.nEq; saved_nBtm = pNew->u.btree.nBtm; |
︙ | ︙ | |||
166027 166028 166029 166030 166031 166032 166033 166034 166035 166036 166037 166038 166039 166040 | }else if( j>=0 && (pTab->aCol[j].colFlags & COLFLAG_VIRTUAL)!=0 ){ pExpr = sqlite3ColumnExpr(pTab, &pTab->aCol[j]); bMaybeNullRow = 0; }else{ continue; } if( sqlite3ExprIsConstant(pExpr) ) continue; p = sqlite3DbMallocRaw(pParse->db, sizeof(IndexedExpr)); if( p==0 ) break; p->pIENext = pParse->pIdxEpr; #ifdef WHERETRACE_ENABLED if( sqlite3WhereTrace & 0x200 ){ sqlite3DebugPrintf("New pParse->pIdxEpr term {%d,%d}\n", iIdxCur, i); if( sqlite3WhereTrace & 0x5000 ) sqlite3ShowExpr(pExpr); | > > > > > > > > > > > > > > | 166617 166618 166619 166620 166621 166622 166623 166624 166625 166626 166627 166628 166629 166630 166631 166632 166633 166634 166635 166636 166637 166638 166639 166640 166641 166642 166643 166644 | }else if( j>=0 && (pTab->aCol[j].colFlags & COLFLAG_VIRTUAL)!=0 ){ pExpr = sqlite3ColumnExpr(pTab, &pTab->aCol[j]); bMaybeNullRow = 0; }else{ continue; } if( sqlite3ExprIsConstant(pExpr) ) continue; if( pExpr->op==TK_FUNCTION ){ /* Functions that might set a subtype should not be replaced by the ** value taken from an expression index since the index omits the ** subtype. https://sqlite.org/forum/forumpost/68d284c86b082c3e */ int n; FuncDef *pDef; sqlite3 *db = pParse->db; assert( ExprUseXList(pExpr) ); n = pExpr->x.pList ? pExpr->x.pList->nExpr : 0; pDef = sqlite3FindFunction(db, pExpr->u.zToken, n, ENC(db), 0); if( pDef==0 || (pDef->funcFlags & SQLITE_RESULT_SUBTYPE)!=0 ){ continue; } } p = sqlite3DbMallocRaw(pParse->db, sizeof(IndexedExpr)); if( p==0 ) break; p->pIENext = pParse->pIdxEpr; #ifdef WHERETRACE_ENABLED if( sqlite3WhereTrace & 0x200 ){ sqlite3DebugPrintf("New pParse->pIdxEpr term {%d,%d}\n", iIdxCur, i); if( sqlite3WhereTrace & 0x5000 ) sqlite3ShowExpr(pExpr); |
︙ | ︙ | |||
166230 166231 166232 166233 166234 166235 166236 | /* Allocate and initialize the WhereInfo structure that will become the ** return value. A single allocation is used to store the WhereInfo ** struct, the contents of WhereInfo.a[], the WhereClause structure ** and the WhereMaskSet structure. Since WhereClause contains an 8-byte ** field (type Bitmask) it must be aligned on an 8-byte boundary on ** some architectures. Hence the ROUND8() below. */ | | > > > | 166834 166835 166836 166837 166838 166839 166840 166841 166842 166843 166844 166845 166846 166847 166848 166849 166850 166851 | /* Allocate and initialize the WhereInfo structure that will become the ** return value. A single allocation is used to store the WhereInfo ** struct, the contents of WhereInfo.a[], the WhereClause structure ** and the WhereMaskSet structure. Since WhereClause contains an 8-byte ** field (type Bitmask) it must be aligned on an 8-byte boundary on ** some architectures. Hence the ROUND8() below. */ nByteWInfo = ROUND8P(sizeof(WhereInfo)); if( nTabList>1 ){ nByteWInfo = ROUND8P(nByteWInfo + (nTabList-1)*sizeof(WhereLevel)); } pWInfo = sqlite3DbMallocRawNN(db, nByteWInfo + sizeof(WhereLoop)); if( db->mallocFailed ){ sqlite3DbFree(db, pWInfo); pWInfo = 0; goto whereBeginError; } pWInfo->pParse = pParse; |
︙ | ︙ | |||
166792 166793 166794 166795 166796 166797 166798 166799 166800 166801 166802 166803 166804 166805 | /* Jump here if malloc fails */ whereBeginError: if( pWInfo ){ pParse->nQueryLoop = pWInfo->savedNQueryLoop; whereInfoFree(db, pWInfo); } return 0; } /* ** Part of sqlite3WhereEnd() will rewrite opcodes to reference the ** index rather than the main table. In SQLITE_DEBUG mode, we want ** to trace those changes if PRAGMA vdbe_addoptrace=on. This routine | > > > > > | 167399 167400 167401 167402 167403 167404 167405 167406 167407 167408 167409 167410 167411 167412 167413 167414 167415 167416 167417 | /* Jump here if malloc fails */ whereBeginError: if( pWInfo ){ pParse->nQueryLoop = pWInfo->savedNQueryLoop; whereInfoFree(db, pWInfo); } #ifdef WHERETRACE_ENABLED /* Prevent harmless compiler warnings about debugging routines ** being declared but never used */ sqlite3ShowWhereLoopList(0); #endif /* WHERETRACE_ENABLED */ return 0; } /* ** Part of sqlite3WhereEnd() will rewrite opcodes to reference the ** index rather than the main table. In SQLITE_DEBUG mode, we want ** to trace those changes if PRAGMA vdbe_addoptrace=on. This routine |
︙ | ︙ | |||
168209 168210 168211 168212 168213 168214 168215 | ** window function - one for the accumulator, another for interim ** results. */ for(pWin=pMWin; pWin; pWin=pWin->pNextWin){ ExprList *pArgs; assert( ExprUseXList(pWin->pOwner) ); assert( pWin->pWFunc!=0 ); pArgs = pWin->pOwner->x.pList; | | | 168821 168822 168823 168824 168825 168826 168827 168828 168829 168830 168831 168832 168833 168834 168835 | ** window function - one for the accumulator, another for interim ** results. */ for(pWin=pMWin; pWin; pWin=pWin->pNextWin){ ExprList *pArgs; assert( ExprUseXList(pWin->pOwner) ); assert( pWin->pWFunc!=0 ); pArgs = pWin->pOwner->x.pList; if( pWin->pWFunc->funcFlags & SQLITE_SUBTYPE ){ selectWindowRewriteEList(pParse, pMWin, pSrc, pArgs, pTab, &pSublist); pWin->iArgCol = (pSublist ? pSublist->nExpr : 0); pWin->bExprArgs = 1; }else{ pWin->iArgCol = (pSublist ? pSublist->nExpr : 0); pSublist = exprListAppendList(pParse, pSublist, pArgs, 0); } |
︙ | ︙ | |||
177851 177852 177853 177854 177855 177856 177857 177858 177859 177860 177861 177862 177863 177864 177865 | int SQLITE_EXTRA_INIT(const char*); rc = SQLITE_EXTRA_INIT(0); } #endif /* Experimentally determine if high-precision floating point is ** available. */ sqlite3Config.bUseLongDouble = hasHighPrecisionDouble(rc); return rc; } /* ** Undo the effects of sqlite3_initialize(). Must not be called while ** there are outstanding database connections or memory allocations or | > > | 178463 178464 178465 178466 178467 178468 178469 178470 178471 178472 178473 178474 178475 178476 178477 178478 178479 | int SQLITE_EXTRA_INIT(const char*); rc = SQLITE_EXTRA_INIT(0); } #endif /* Experimentally determine if high-precision floating point is ** available. */ #ifndef SQLITE_OMIT_WSD sqlite3Config.bUseLongDouble = hasHighPrecisionDouble(rc); #endif return rc; } /* ** Undo the effects of sqlite3_initialize(). Must not be called while ** there are outstanding database connections or memory allocations or |
︙ | ︙ | |||
179381 179382 179383 179384 179385 179386 179387 | ){ return SQLITE_MISUSE_BKPT; } assert( SQLITE_FUNC_CONSTANT==SQLITE_DETERMINISTIC ); assert( SQLITE_FUNC_DIRECT==SQLITE_DIRECTONLY ); extraFlags = enc & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY| | | | 179995 179996 179997 179998 179999 180000 180001 180002 180003 180004 180005 180006 180007 180008 180009 | ){ return SQLITE_MISUSE_BKPT; } assert( SQLITE_FUNC_CONSTANT==SQLITE_DETERMINISTIC ); assert( SQLITE_FUNC_DIRECT==SQLITE_DIRECTONLY ); extraFlags = enc & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY| SQLITE_SUBTYPE|SQLITE_INNOCUOUS|SQLITE_RESULT_SUBTYPE); enc &= (SQLITE_FUNC_ENCMASK|SQLITE_ANY); /* The SQLITE_INNOCUOUS flag is the same bit as SQLITE_FUNC_UNSAFE. But ** the meaning is inverted. So flip the bit. */ assert( SQLITE_FUNC_UNSAFE==SQLITE_INNOCUOUS ); extraFlags ^= SQLITE_FUNC_UNSAFE; /* tag-20230109-1 */ |
︙ | ︙ | |||
182126 182127 182128 182129 182130 182131 182132 182133 182134 182135 182136 182137 182138 182139 | *piValue = Tuning(-id); }else{ rc = SQLITE_NOTFOUND; } break; } #endif } va_end(ap); #endif /* SQLITE_UNTESTABLE */ return rc; } /* | > > > > > > > > > > > > > > > > > > > > > > | 182740 182741 182742 182743 182744 182745 182746 182747 182748 182749 182750 182751 182752 182753 182754 182755 182756 182757 182758 182759 182760 182761 182762 182763 182764 182765 182766 182767 182768 182769 182770 182771 182772 182773 182774 182775 | *piValue = Tuning(-id); }else{ rc = SQLITE_NOTFOUND; } break; } #endif /* sqlite3_test_control(SQLITE_TESTCTRL_JSON_SELFCHECK, &onOff); ** ** Activate or deactivate validation of JSONB that is generated from ** text. Off by default, as the validation is slow. Validation is ** only available if compiled using SQLITE_DEBUG. ** ** If onOff is initially 1, then turn it on. If onOff is initially ** off, turn it off. If onOff is initially -1, then change onOff ** to be the current setting. */ case SQLITE_TESTCTRL_JSON_SELFCHECK: { #if defined(SQLITE_DEBUG) && !defined(SQLITE_OMIT_WSD) int *pOnOff = va_arg(ap, int*); if( *pOnOff<0 ){ *pOnOff = sqlite3Config.bJsonSelfcheck; }else{ sqlite3Config.bJsonSelfcheck = (u8)((*pOnOff)&0xff); } #endif break; } } va_end(ap); #endif /* SQLITE_UNTESTABLE */ return rc; } /* |
︙ | ︙ | |||
187832 187833 187834 187835 187836 187837 187838 | return 0; } /* ** Implementation of the xIntegrity() method on the FTS3/FTS4 virtual ** table. */ | | > > > > > > > > > < > | | | 188468 188469 188470 188471 188472 188473 188474 188475 188476 188477 188478 188479 188480 188481 188482 188483 188484 188485 188486 188487 188488 188489 188490 188491 188492 188493 188494 188495 188496 188497 188498 188499 188500 188501 188502 188503 188504 188505 188506 188507 188508 188509 188510 188511 | return 0; } /* ** Implementation of the xIntegrity() method on the FTS3/FTS4 virtual ** table. */ static int fts3Integrity( sqlite3_vtab *pVtab, /* The virtual table to be checked */ const char *zSchema, /* Name of schema in which pVtab lives */ const char *zTabname, /* Name of the pVTab table */ int isQuick, /* True if this is a quick_check */ char **pzErr /* Write error message here */ ){ Fts3Table *p = (Fts3Table*)pVtab; char *zSql; int rc; char *zErr = 0; assert( pzErr!=0 ); assert( *pzErr==0 ); UNUSED_PARAMETER(isQuick); zSql = sqlite3_mprintf( "INSERT INTO \"%w\".\"%w\"(\"%w\") VALUES('integrity-check');", zSchema, zTabname, zTabname); if( zSql==0 ){ return SQLITE_NOMEM; } rc = sqlite3_exec(p->db, zSql, 0, 0, &zErr); sqlite3_free(zSql); if( (rc&0xff)==SQLITE_CORRUPT ){ *pzErr = sqlite3_mprintf("malformed inverted index for FTS%d table %s.%s", p->bFts4 ? 4 : 3, zSchema, zTabname); }else if( rc!=SQLITE_OK ){ *pzErr = sqlite3_mprintf("unable to validate the inverted index for" " FTS%d table %s.%s: %s", p->bFts4 ? 4 : 3, zSchema, zTabname, zErr); } sqlite3_free(zErr); return SQLITE_OK; } |
︙ | ︙ | |||
202531 202532 202533 202534 202535 202536 202537 | ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** | | | > > > > > > > > > > > > > > > > > > | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > | | > > | 203176 203177 203178 203179 203180 203181 203182 203183 203184 203185 203186 203187 203188 203189 203190 203191 203192 203193 203194 203195 203196 203197 203198 203199 203200 203201 203202 203203 203204 203205 203206 203207 203208 203209 203210 203211 203212 203213 203214 203215 203216 203217 203218 203219 203220 203221 203222 203223 203224 203225 203226 203227 203228 203229 203230 203231 203232 203233 203234 203235 203236 203237 203238 203239 203240 203241 203242 203243 203244 203245 203246 203247 203248 203249 203250 203251 203252 203253 203254 203255 203256 203257 203258 203259 203260 203261 203262 203263 203264 203265 203266 203267 203268 203269 203270 203271 203272 203273 203274 203275 203276 203277 203278 203279 203280 203281 203282 203283 203284 203285 203286 203287 203288 203289 203290 203291 203292 203293 203294 203295 203296 203297 203298 203299 203300 203301 203302 203303 203304 203305 203306 203307 203308 203309 203310 203311 203312 203313 203314 203315 203316 203317 203318 203319 203320 203321 203322 203323 203324 203325 203326 203327 203328 203329 203330 203331 203332 203333 203334 203335 203336 203337 203338 203339 203340 203341 203342 203343 203344 203345 203346 203347 203348 203349 203350 203351 203352 203353 203354 203355 203356 203357 203358 203359 203360 203361 | ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** SQLite JSON functions. ** ** This file began as an extension in ext/misc/json1.c in 2015. That ** extension proved so useful that it has now been moved into the core. ** ** The original design stored all JSON as pure text, canonical RFC-8259. ** Support for JSON-5 extensions was added with version 3.42.0 (2023-05-16). ** All generated JSON text still conforms strictly to RFC-8259, but text ** with JSON-5 extensions is accepted as input. ** ** Beginning with version 3.45.0 (circa 2024-01-01), these routines also ** accept BLOB values that have JSON encoded using a binary representation ** called "JSONB". The name JSONB comes from PostgreSQL, however the on-disk ** format SQLite JSONB is completely different and incompatible with ** PostgreSQL JSONB. ** ** Decoding and interpreting JSONB is still O(N) where N is the size of ** the input, the same as text JSON. However, the constant of proportionality ** for JSONB is much smaller due to faster parsing. The size of each ** element in JSONB is encoded in its header, so there is no need to search ** for delimiters using persnickety syntax rules. JSONB seems to be about ** 3x faster than text JSON as a result. JSONB is also tends to be slightly ** smaller than text JSON, by 5% or 10%, but there are corner cases where ** JSONB can be slightly larger. So you are not far mistaken to say that ** a JSONB blob is the same size as the equivalent RFC-8259 text. ** ** ** THE JSONB ENCODING: ** ** Every JSON element is encoded in JSONB as a header and a payload. ** The header is between 1 and 9 bytes in size. The payload is zero ** or more bytes. ** ** The lower 4 bits of the first byte of the header determines the ** element type: ** ** 0: NULL ** 1: TRUE ** 2: FALSE ** 3: INT -- RFC-8259 integer literal ** 4: INT5 -- JSON5 integer literal ** 5: FLOAT -- RFC-8259 floating point literal ** 6: FLOAT5 -- JSON5 floating point literal ** 7: TEXT -- Text literal acceptable to both SQL and JSON ** 8: TEXTJ -- Text containing RFC-8259 escapes ** 9: TEXT5 -- Text containing JSON5 and/or RFC-8259 escapes ** 10: TEXTRAW -- Text containing unescaped syntax characters ** 11: ARRAY ** 12: OBJECT ** ** The other three possible values (13-15) are reserved for future ** enhancements. ** ** The upper 4 bits of the first byte determine the size of the header ** and sometimes also the size of the payload. If X is the first byte ** of the element and if X>>4 is between 0 and 11, then the payload ** will be that many bytes in size and the header is exactly one byte ** in size. Other four values for X>>4 (12-15) indicate that the header ** is more than one byte in size and that the payload size is determined ** by the remainder of the header, interpreted as a unsigned big-endian ** integer. ** ** Value of X>>4 Size integer Total header size ** ------------- -------------------- ----------------- ** 12 1 byte (0-255) 2 ** 13 2 byte (0-65535) 3 ** 14 4 byte (0-4294967295) 5 ** 15 8 byte (0-1.8e19) 9 ** ** The payload size need not be expressed in its minimal form. For example, ** if the payload size is 10, the size can be expressed in any of 5 different ** ways: (1) (X>>4)==10, (2) (X>>4)==12 following by on 0x0a byte, ** (3) (X>>4)==13 followed by 0x00 and 0x0a, (4) (X>>4)==14 followed by ** 0x00 0x00 0x00 0x0a, or (5) (X>>4)==15 followed by 7 bytes of 0x00 and ** a single byte of 0x0a. The shorter forms are preferred, of course, but ** sometimes when generating JSONB, the payload size is not known in advance ** and it is convenient to reserve sufficient header space to cover the ** largest possible payload size and then come back later and patch up ** the size when it becomes known, resulting in a non-minimal encoding. ** ** The value (X>>4)==15 is not actually used in the current implementation ** (as SQLite is currently unable handle BLOBs larger than about 2GB) ** but is included in the design to allow for future enhancements. ** ** The payload follows the header. NULL, TRUE, and FALSE have no payload and ** their payload size must always be zero. The payload for INT, INT5, ** FLOAT, FLOAT5, TEXT, TEXTJ, TEXT5, and TEXTROW is text. Note that the ** "..." or '...' delimiters are omitted from the various text encodings. ** The payload for ARRAY and OBJECT is a list of additional elements that ** are the content for the array or object. The payload for an OBJECT ** must be an even number of elements. The first element of each pair is ** the label and must be of type TEXT, TEXTJ, TEXT5, or TEXTRAW. ** ** A valid JSONB blob consists of a single element, as described above. ** Usually this will be an ARRAY or OBJECT element which has many more ** elements as its content. But the overall blob is just a single element. ** ** Input validation for JSONB blobs simply checks that the element type ** code is between 0 and 12 and that the total size of the element ** (header plus payload) is the same as the size of the BLOB. If those ** checks are true, the BLOB is assumed to be JSONB and processing continues. ** Errors are only raised if some other miscoding is discovered during ** processing. ** ** Additional information can be found in the doc/jsonb.md file of the ** canonical SQLite source tree. */ #ifndef SQLITE_OMIT_JSON /* #include "sqliteInt.h" */ /* JSONB element types */ #define JSONB_NULL 0 /* "null" */ #define JSONB_TRUE 1 /* "true" */ #define JSONB_FALSE 2 /* "false" */ #define JSONB_INT 3 /* integer acceptable to JSON and SQL */ #define JSONB_INT5 4 /* integer in 0x000 notation */ #define JSONB_FLOAT 5 /* float acceptable to JSON and SQL */ #define JSONB_FLOAT5 6 /* float with JSON5 extensions */ #define JSONB_TEXT 7 /* Text compatible with both JSON and SQL */ #define JSONB_TEXTJ 8 /* Text with JSON escapes */ #define JSONB_TEXT5 9 /* Text with JSON-5 escape */ #define JSONB_TEXTRAW 10 /* SQL text that needs escaping for JSON */ #define JSONB_ARRAY 11 /* An array */ #define JSONB_OBJECT 12 /* An object */ /* Human-readable names for the JSONB values. The index for each ** string must correspond to the JSONB_* integer above. */ static const char * const jsonbType[] = { "null", "true", "false", "integer", "integer", "real", "real", "text", "text", "text", "text", "array", "object", "", "", "", "" }; /* ** Growing our own isspace() routine this way is twice as fast as ** the library isspace() function, resulting in a 7% overall performance ** increase for the text-JSON parser. (Ubuntu14.10 gcc 4.8.4 x64 with -Os). */ static const char jsonIsSpace[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }; #define jsonIsspace(x) (jsonIsSpace[(unsigned char)x]) /* ** The set of all space characters recognized by jsonIsspace(). ** Useful as the second argument to strspn(). */ static const char jsonSpaces[] = "\011\012\015\040"; /* ** Characters that are special to JSON. Control characters, ** '"' and '\\' and '\''. Actually, '\'' is not special to ** canonical JSON, but it is special in JSON-5, so we include ** it in the set of special characters. */ static const char jsonIsOk[256] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, |
︙ | ︙ | |||
202595 202596 202597 202598 202599 202600 202601 | 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 }; | < < < < < < < | | > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > | < < < < < < < < | < < < | | | < < < < < | > > < < < < < < | | < < | | < | | | < < < < < < < < < < < < < < < < < < < < < | < | | > < | | | < < < < < < < < < > > | | | | | | | | > | | | | < | | | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | | | | | | | | | | | | | | | | | | | | | | | < < < < < < | < | | > > > | > > > | < > | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > | | | > > < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | | 203369 203370 203371 203372 203373 203374 203375 203376 203377 203378 203379 203380 203381 203382 203383 203384 203385 203386 203387 203388 203389 203390 203391 203392 203393 203394 203395 203396 203397 203398 203399 203400 203401 203402 203403 203404 203405 203406 203407 203408 203409 203410 203411 203412 203413 203414 203415 203416 203417 203418 203419 203420 203421 203422 203423 203424 203425 203426 203427 203428 203429 203430 203431 203432 203433 203434 203435 203436 203437 203438 203439 203440 203441 203442 203443 203444 203445 203446 203447 203448 203449 203450 203451 203452 203453 203454 203455 203456 203457 203458 203459 203460 203461 203462 203463 203464 203465 203466 203467 203468 203469 203470 203471 203472 203473 203474 203475 203476 203477 203478 203479 203480 203481 203482 203483 203484 203485 203486 203487 203488 203489 203490 203491 203492 203493 203494 203495 203496 203497 203498 203499 203500 203501 203502 203503 203504 203505 203506 203507 203508 203509 203510 203511 203512 203513 203514 203515 203516 203517 203518 203519 203520 203521 203522 203523 203524 203525 203526 203527 203528 203529 203530 203531 203532 203533 203534 203535 203536 203537 203538 203539 203540 203541 203542 203543 203544 203545 203546 203547 203548 203549 203550 203551 203552 203553 203554 203555 203556 203557 203558 203559 203560 203561 203562 203563 203564 203565 203566 203567 203568 203569 203570 203571 203572 203573 203574 203575 203576 203577 203578 203579 203580 203581 203582 203583 203584 203585 203586 203587 203588 203589 203590 203591 203592 203593 203594 203595 203596 203597 203598 203599 203600 203601 203602 203603 203604 203605 203606 203607 203608 203609 203610 203611 203612 203613 203614 203615 203616 203617 203618 203619 203620 203621 203622 203623 203624 203625 203626 203627 203628 203629 203630 203631 203632 203633 203634 203635 203636 203637 203638 203639 203640 203641 203642 203643 203644 203645 203646 203647 203648 203649 203650 203651 203652 203653 203654 203655 203656 203657 203658 203659 203660 203661 203662 203663 203664 203665 203666 203667 203668 203669 203670 203671 203672 203673 203674 203675 203676 203677 203678 203679 203680 203681 203682 203683 203684 203685 203686 203687 203688 203689 203690 203691 203692 203693 203694 203695 203696 203697 203698 203699 203700 203701 203702 203703 203704 203705 203706 203707 203708 203709 203710 203711 203712 203713 203714 203715 203716 203717 203718 203719 203720 203721 203722 203723 203724 203725 203726 203727 203728 203729 203730 203731 203732 203733 203734 203735 203736 203737 203738 203739 203740 203741 203742 203743 203744 203745 203746 203747 203748 203749 203750 203751 203752 203753 203754 203755 203756 203757 203758 203759 203760 203761 203762 203763 203764 203765 203766 203767 203768 203769 203770 203771 203772 203773 203774 203775 203776 203777 203778 203779 203780 203781 203782 203783 203784 203785 203786 203787 203788 203789 203790 203791 203792 203793 203794 203795 203796 203797 203798 203799 203800 203801 203802 203803 203804 203805 203806 203807 203808 203809 203810 203811 203812 203813 203814 203815 203816 203817 203818 203819 203820 203821 203822 203823 203824 203825 203826 203827 203828 203829 203830 203831 203832 203833 203834 203835 203836 203837 203838 203839 203840 203841 203842 203843 203844 203845 203846 203847 203848 203849 203850 203851 203852 203853 203854 203855 203856 203857 203858 203859 203860 203861 203862 203863 203864 203865 203866 203867 203868 203869 203870 203871 203872 203873 203874 203875 203876 203877 203878 203879 203880 203881 203882 203883 203884 203885 203886 203887 203888 203889 203890 203891 | 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 }; /* Objects */ typedef struct JsonCache JsonCache; typedef struct JsonString JsonString; typedef struct JsonParse JsonParse; /* ** Magic number used for the JSON parse cache in sqlite3_get_auxdata() */ #define JSON_CACHE_ID (-429938) /* Cache entry */ #define JSON_CACHE_SIZE 4 /* Max number of cache entries */ /* ** jsonUnescapeOneChar() returns this invalid code point if it encounters ** a syntax error. */ #define JSON_INVALID_CHAR 0x99999 /* A cache mapping JSON text into JSONB blobs. ** ** Each cache entry is a JsonParse object with the following restrictions: ** ** * The bReadOnly flag must be set ** ** * The aBlob[] array must be owned by the JsonParse object. In other ** words, nBlobAlloc must be non-zero. ** ** * eEdit and delta must be zero. ** ** * zJson must be an RCStr. In other words bJsonIsRCStr must be true. */ struct JsonCache { sqlite3 *db; /* Database connection */ int nUsed; /* Number of active entries in the cache */ JsonParse *a[JSON_CACHE_SIZE]; /* One line for each cache entry */ }; /* An instance of this object represents a JSON string ** under construction. Really, this is a generic string accumulator ** that can be and is used to create strings other than JSON. ** ** If the generated string is longer than will fit into the zSpace[] buffer, ** then it will be an RCStr string. This aids with caching of large ** JSON strings. */ struct JsonString { sqlite3_context *pCtx; /* Function context - put error messages here */ char *zBuf; /* Append JSON content here */ u64 nAlloc; /* Bytes of storage available in zBuf[] */ u64 nUsed; /* Bytes of zBuf[] currently used */ u8 bStatic; /* True if zBuf is static space */ u8 eErr; /* True if an error has been encountered */ char zSpace[100]; /* Initial static space */ }; /* Allowed values for JsonString.eErr */ #define JSTRING_OOM 0x01 /* Out of memory */ #define JSTRING_MALFORMED 0x02 /* Malformed JSONB */ #define JSTRING_ERR 0x04 /* Error already sent to sqlite3_result */ /* The "subtype" set for text JSON values passed through using ** sqlite3_result_subtype() and sqlite3_value_subtype(). */ #define JSON_SUBTYPE 74 /* Ascii for "J" */ /* ** Bit values for the flags passed into various SQL function implementations ** via the sqlite3_user_data() value. */ #define JSON_JSON 0x01 /* Result is always JSON */ #define JSON_SQL 0x02 /* Result is always SQL */ #define JSON_ABPATH 0x03 /* Allow abbreviated JSON path specs */ #define JSON_ISSET 0x04 /* json_set(), not json_insert() */ #define JSON_BLOB 0x08 /* Use the BLOB output format */ /* A parsed JSON value. Lifecycle: ** ** 1. JSON comes in and is parsed into a JSONB value in aBlob. The ** original text is stored in zJson. This step is skipped if the ** input is JSONB instead of text JSON. ** ** 2. The aBlob[] array is searched using the JSON path notation, if needed. ** ** 3. Zero or more changes are made to aBlob[] (via json_remove() or ** json_replace() or json_patch() or similar). ** ** 4. New JSON text is generated from the aBlob[] for output. This step ** is skipped if the function is one of the jsonb_* functions that ** returns JSONB instead of text JSON. */ struct JsonParse { u8 *aBlob; /* JSONB representation of JSON value */ u32 nBlob; /* Bytes of aBlob[] actually used */ u32 nBlobAlloc; /* Bytes allocated to aBlob[]. 0 if aBlob is external */ char *zJson; /* Json text used for parsing */ sqlite3 *db; /* The database connection to which this object belongs */ int nJson; /* Length of the zJson string in bytes */ u32 nJPRef; /* Number of references to this object */ u32 iErr; /* Error location in zJson[] */ u16 iDepth; /* Nesting depth */ u8 nErr; /* Number of errors seen */ u8 oom; /* Set to true if out of memory */ u8 bJsonIsRCStr; /* True if zJson is an RCStr */ u8 hasNonstd; /* True if input uses non-standard features like JSON5 */ u8 bReadOnly; /* Do not modify. */ /* Search and edit information. See jsonLookupStep() */ u8 eEdit; /* Edit operation to apply */ int delta; /* Size change due to the edit */ u32 nIns; /* Number of bytes to insert */ u32 iLabel; /* Location of label if search landed on an object value */ u8 *aIns; /* Content to be inserted */ }; /* Allowed values for JsonParse.eEdit */ #define JEDIT_DEL 1 /* Delete if exists */ #define JEDIT_REPL 2 /* Overwrite if exists */ #define JEDIT_INS 3 /* Insert if not exists */ #define JEDIT_SET 4 /* Insert or overwrite */ /* ** Maximum nesting depth of JSON for this implementation. ** ** This limit is needed to avoid a stack overflow in the recursive ** descent parser. A depth of 1000 is far deeper than any sane JSON ** should go. Historical note: This limit was 2000 prior to version 3.42.0 */ #ifndef SQLITE_JSON_MAX_DEPTH # define JSON_MAX_DEPTH 1000 #else # define JSON_MAX_DEPTH SQLITE_JSON_MAX_DEPTH #endif /* ** Allowed values for the flgs argument to jsonParseFuncArg(); */ #define JSON_EDITABLE 0x01 /* Generate a writable JsonParse object */ #define JSON_KEEPERROR 0x02 /* Return non-NULL even if there is an error */ /************************************************************************** ** Forward references **************************************************************************/ static void jsonReturnStringAsBlob(JsonString*); static int jsonFuncArgMightBeBinary(sqlite3_value *pJson); static u32 jsonTranslateBlobToText(const JsonParse*,u32,JsonString*); static void jsonReturnParse(sqlite3_context*,JsonParse*); static JsonParse *jsonParseFuncArg(sqlite3_context*,sqlite3_value*,u32); static void jsonParseFree(JsonParse*); static u32 jsonbPayloadSize(const JsonParse*, u32, u32*); static u32 jsonUnescapeOneChar(const char*, u32, u32*); /************************************************************************** ** Utility routines for dealing with JsonCache objects **************************************************************************/ /* ** Free a JsonCache object. */ static void jsonCacheDelete(JsonCache *p){ int i; for(i=0; i<p->nUsed; i++){ jsonParseFree(p->a[i]); } sqlite3DbFree(p->db, p); } static void jsonCacheDeleteGeneric(void *p){ jsonCacheDelete((JsonCache*)p); } /* ** Insert a new entry into the cache. If the cache is full, expel ** the least recently used entry. Return SQLITE_OK on success or a ** result code otherwise. ** ** Cache entries are stored in age order, oldest first. */ static int jsonCacheInsert( sqlite3_context *ctx, /* The SQL statement context holding the cache */ JsonParse *pParse /* The parse object to be added to the cache */ ){ JsonCache *p; assert( pParse->zJson!=0 ); assert( pParse->bJsonIsRCStr ); assert( pParse->delta==0 ); p = sqlite3_get_auxdata(ctx, JSON_CACHE_ID); if( p==0 ){ sqlite3 *db = sqlite3_context_db_handle(ctx); p = sqlite3DbMallocZero(db, sizeof(*p)); if( p==0 ) return SQLITE_NOMEM; p->db = db; sqlite3_set_auxdata(ctx, JSON_CACHE_ID, p, jsonCacheDeleteGeneric); p = sqlite3_get_auxdata(ctx, JSON_CACHE_ID); if( p==0 ) return SQLITE_NOMEM; } if( p->nUsed >= JSON_CACHE_SIZE ){ jsonParseFree(p->a[0]); memmove(p->a, &p->a[1], (JSON_CACHE_SIZE-1)*sizeof(p->a[0])); p->nUsed = JSON_CACHE_SIZE-1; } assert( pParse->nBlobAlloc>0 ); pParse->eEdit = 0; pParse->nJPRef++; pParse->bReadOnly = 1; p->a[p->nUsed] = pParse; p->nUsed++; return SQLITE_OK; } /* ** Search for a cached translation the json text supplied by pArg. Return ** the JsonParse object if found. Return NULL if not found. ** ** When a match if found, the matching entry is moved to become the ** most-recently used entry if it isn't so already. ** ** The JsonParse object returned still belongs to the Cache and might ** be deleted at any moment. If the caller whants the JsonParse to ** linger, it needs to increment the nPJRef reference counter. */ static JsonParse *jsonCacheSearch( sqlite3_context *ctx, /* The SQL statement context holding the cache */ sqlite3_value *pArg /* Function argument containing SQL text */ ){ JsonCache *p; int i; const char *zJson; int nJson; if( sqlite3_value_type(pArg)!=SQLITE_TEXT ){ return 0; } zJson = (const char*)sqlite3_value_text(pArg); if( zJson==0 ) return 0; nJson = sqlite3_value_bytes(pArg); p = sqlite3_get_auxdata(ctx, JSON_CACHE_ID); if( p==0 ){ return 0; } for(i=0; i<p->nUsed; i++){ if( p->a[i]->zJson==zJson ) break; } if( i>=p->nUsed ){ for(i=0; i<p->nUsed; i++){ if( p->a[i]->nJson!=nJson ) continue; if( memcmp(p->a[i]->zJson, zJson, nJson)==0 ) break; } } if( i<p->nUsed ){ if( i<p->nUsed-1 ){ /* Make the matching entry the most recently used entry */ JsonParse *tmp = p->a[i]; memmove(&p->a[i], &p->a[i+1], (p->nUsed-i-1)*sizeof(tmp)); p->a[p->nUsed-1] = tmp; i = p->nUsed - 1; } assert( p->a[i]->delta==0 ); return p->a[i]; }else{ return 0; } } /************************************************************************** ** Utility routines for dealing with JsonString objects **************************************************************************/ /* Turn uninitialized bulk memory into a valid JsonString object ** holding a zero-length string. */ static void jsonStringZero(JsonString *p){ p->zBuf = p->zSpace; p->nAlloc = sizeof(p->zSpace); p->nUsed = 0; p->bStatic = 1; } /* Initialize the JsonString object */ static void jsonStringInit(JsonString *p, sqlite3_context *pCtx){ p->pCtx = pCtx; p->eErr = 0; jsonStringZero(p); } /* Free all allocated memory and reset the JsonString object back to its ** initial state. */ static void jsonStringReset(JsonString *p){ if( !p->bStatic ) sqlite3RCStrUnref(p->zBuf); jsonStringZero(p); } /* Report an out-of-memory (OOM) condition */ static void jsonStringOom(JsonString *p){ p->eErr |= JSTRING_OOM; if( p->pCtx ) sqlite3_result_error_nomem(p->pCtx); jsonStringReset(p); } /* Enlarge pJson->zBuf so that it can hold at least N more bytes. ** Return zero on success. Return non-zero on an OOM error */ static int jsonStringGrow(JsonString *p, u32 N){ u64 nTotal = N<p->nAlloc ? p->nAlloc*2 : p->nAlloc+N+10; char *zNew; if( p->bStatic ){ if( p->eErr ) return 1; zNew = sqlite3RCStrNew(nTotal); if( zNew==0 ){ jsonStringOom(p); return SQLITE_NOMEM; } memcpy(zNew, p->zBuf, (size_t)p->nUsed); p->zBuf = zNew; p->bStatic = 0; }else{ p->zBuf = sqlite3RCStrResize(p->zBuf, nTotal); if( p->zBuf==0 ){ p->eErr |= JSTRING_OOM; jsonStringZero(p); return SQLITE_NOMEM; } } p->nAlloc = nTotal; return SQLITE_OK; } /* Append N bytes from zIn onto the end of the JsonString string. */ static SQLITE_NOINLINE void jsonStringExpandAndAppend( JsonString *p, const char *zIn, u32 N ){ assert( N>0 ); if( jsonStringGrow(p,N) ) return; memcpy(p->zBuf+p->nUsed, zIn, N); p->nUsed += N; } static void jsonAppendRaw(JsonString *p, const char *zIn, u32 N){ if( N==0 ) return; if( N+p->nUsed >= p->nAlloc ){ jsonStringExpandAndAppend(p,zIn,N); }else{ memcpy(p->zBuf+p->nUsed, zIn, N); p->nUsed += N; } } static void jsonAppendRawNZ(JsonString *p, const char *zIn, u32 N){ assert( N>0 ); if( N+p->nUsed >= p->nAlloc ){ jsonStringExpandAndAppend(p,zIn,N); }else{ memcpy(p->zBuf+p->nUsed, zIn, N); p->nUsed += N; } } /* Append formatted text (not to exceed N bytes) to the JsonString. */ static void jsonPrintf(int N, JsonString *p, const char *zFormat, ...){ va_list ap; if( (p->nUsed + N >= p->nAlloc) && jsonStringGrow(p, N) ) return; va_start(ap, zFormat); sqlite3_vsnprintf(N, p->zBuf+p->nUsed, zFormat, ap); va_end(ap); p->nUsed += (int)strlen(p->zBuf+p->nUsed); } /* Append a single character */ static SQLITE_NOINLINE void jsonAppendCharExpand(JsonString *p, char c){ if( jsonStringGrow(p,1) ) return; p->zBuf[p->nUsed++] = c; } static void jsonAppendChar(JsonString *p, char c){ if( p->nUsed>=p->nAlloc ){ jsonAppendCharExpand(p,c); }else{ p->zBuf[p->nUsed++] = c; } } /* Make sure there is a zero terminator on p->zBuf[] ** ** Return true on success. Return false if an OOM prevents this ** from happening. */ static int jsonStringTerminate(JsonString *p){ jsonAppendChar(p, 0); p->nUsed--; return p->eErr==0; } /* Append a comma separator to the output buffer, if the previous ** character is not '[' or '{'. */ static void jsonAppendSeparator(JsonString *p){ char c; if( p->nUsed==0 ) return; c = p->zBuf[p->nUsed-1]; if( c=='[' || c=='{' ) return; jsonAppendChar(p, ','); } /* Append the N-byte string in zIn to the end of the JsonString string ** under construction. Enclose the string in double-quotes ("...") and ** escape any double-quotes or backslash characters contained within the ** string. ** ** This routine is a high-runner. There is a measurable performance ** increase associated with unwinding the jsonIsOk[] loop. */ static void jsonAppendString(JsonString *p, const char *zIn, u32 N){ u32 k; u8 c; const u8 *z = (const u8*)zIn; if( z==0 ) return; if( (N+p->nUsed+2 >= p->nAlloc) && jsonStringGrow(p,N+2)!=0 ) return; p->zBuf[p->nUsed++] = '"'; while( 1 /*exit-by-break*/ ){ k = 0; /* The following while() is the 4-way unwound equivalent of ** ** while( k<N && jsonIsOk[z[k]] ){ k++; } */ while( 1 /* Exit by break */ ){ if( k+3>=N ){ while( k<N && jsonIsOk[z[k]] ){ k++; } break; } if( !jsonIsOk[z[k]] ){ break; } if( !jsonIsOk[z[k+1]] ){ k += 1; break; } if( !jsonIsOk[z[k+2]] ){ k += 2; break; } if( !jsonIsOk[z[k+3]] ){ k += 3; break; }else{ k += 4; } } if( k>=N ){ if( k>0 ){ memcpy(&p->zBuf[p->nUsed], z, k); p->nUsed += k; } break; } if( k>0 ){ memcpy(&p->zBuf[p->nUsed], z, k); p->nUsed += k; z += k; N -= k; } c = z[0]; if( c=='"' || c=='\\' ){ json_simple_escape: if( (p->nUsed+N+3 > p->nAlloc) && jsonStringGrow(p,N+3)!=0 ) return; p->zBuf[p->nUsed++] = '\\'; p->zBuf[p->nUsed++] = c; }else if( c=='\'' ){ p->zBuf[p->nUsed++] = c; }else{ static const char aSpecial[] = { 0, 0, 0, 0, 0, 0, 0, 0, 'b', 't', 'n', 0, 'f', 'r', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; assert( sizeof(aSpecial)==32 ); assert( aSpecial['\b']=='b' ); assert( aSpecial['\f']=='f' ); assert( aSpecial['\n']=='n' ); assert( aSpecial['\r']=='r' ); assert( aSpecial['\t']=='t' ); assert( c>=0 && c<sizeof(aSpecial) ); if( aSpecial[c] ){ c = aSpecial[c]; goto json_simple_escape; } if( (p->nUsed+N+7 > p->nAlloc) && jsonStringGrow(p,N+7)!=0 ) return; p->zBuf[p->nUsed++] = '\\'; p->zBuf[p->nUsed++] = 'u'; p->zBuf[p->nUsed++] = '0'; p->zBuf[p->nUsed++] = '0'; p->zBuf[p->nUsed++] = "0123456789abcdef"[c>>4]; p->zBuf[p->nUsed++] = "0123456789abcdef"[c&0xf]; } z++; N--; } p->zBuf[p->nUsed++] = '"'; assert( p->nUsed<p->nAlloc ); } /* ** Append an sqlite3_value (such as a function parameter) to the JSON ** string under construction in p. */ static void jsonAppendSqlValue( JsonString *p, /* Append to this JSON string */ sqlite3_value *pValue /* Value to append */ ){ switch( sqlite3_value_type(pValue) ){ case SQLITE_NULL: { jsonAppendRawNZ(p, "null", 4); break; |
︙ | ︙ | |||
203100 203101 203102 203103 203104 203105 203106 | jsonAppendRaw(p, z, n); }else{ jsonAppendString(p, z, n); } break; } default: { | > > > > > > | | | | | | > > > > | > > > > > > | > > > | | > > | > > > > > > > > > | | > | | > > | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > | > | | > | | < < < < > | | | | | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < | < < < < < < < < < < < < < < > | | > < | | | > | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 203907 203908 203909 203910 203911 203912 203913 203914 203915 203916 203917 203918 203919 203920 203921 203922 203923 203924 203925 203926 203927 203928 203929 203930 203931 203932 203933 203934 203935 203936 203937 203938 203939 203940 203941 203942 203943 203944 203945 203946 203947 203948 203949 203950 203951 203952 203953 203954 203955 203956 203957 203958 203959 203960 203961 203962 203963 203964 203965 203966 203967 203968 203969 203970 203971 203972 203973 203974 203975 203976 203977 203978 203979 203980 203981 203982 203983 203984 203985 203986 203987 203988 203989 203990 203991 203992 203993 203994 203995 203996 203997 203998 203999 204000 204001 204002 204003 204004 204005 204006 204007 204008 204009 204010 204011 204012 204013 204014 204015 204016 204017 204018 204019 204020 204021 204022 204023 204024 204025 204026 204027 204028 204029 204030 204031 204032 204033 204034 204035 204036 204037 204038 204039 204040 204041 204042 204043 204044 204045 204046 204047 204048 204049 204050 204051 204052 204053 204054 204055 204056 204057 | jsonAppendRaw(p, z, n); }else{ jsonAppendString(p, z, n); } break; } default: { if( jsonFuncArgMightBeBinary(pValue) ){ JsonParse px; memset(&px, 0, sizeof(px)); px.aBlob = (u8*)sqlite3_value_blob(pValue); px.nBlob = sqlite3_value_bytes(pValue); jsonTranslateBlobToText(&px, 0, p); }else if( p->eErr==0 ){ sqlite3_result_error(p->pCtx, "JSON cannot hold BLOB values", -1); p->eErr = JSTRING_ERR; jsonStringReset(p); } break; } } } /* Make the text in p (which is probably a generated JSON text string) ** the result of the SQL function. ** ** The JsonString is reset. ** ** If pParse and ctx are both non-NULL, then the SQL string in p is ** loaded into the zJson field of the pParse object as a RCStr and the ** pParse is added to the cache. */ static void jsonReturnString( JsonString *p, /* String to return */ JsonParse *pParse, /* JSONB source or NULL */ sqlite3_context *ctx /* Where to cache */ ){ assert( (pParse!=0)==(ctx!=0) ); assert( ctx==0 || ctx==p->pCtx ); if( p->eErr==0 ){ int flags = SQLITE_PTR_TO_INT(sqlite3_user_data(p->pCtx)); if( flags & JSON_BLOB ){ jsonReturnStringAsBlob(p); }else if( p->bStatic ){ sqlite3_result_text64(p->pCtx, p->zBuf, p->nUsed, SQLITE_TRANSIENT, SQLITE_UTF8); }else if( jsonStringTerminate(p) ){ if( pParse && pParse->bJsonIsRCStr==0 && pParse->nBlobAlloc>0 ){ int rc; pParse->zJson = sqlite3RCStrRef(p->zBuf); pParse->nJson = p->nUsed; pParse->bJsonIsRCStr = 1; rc = jsonCacheInsert(ctx, pParse); if( rc==SQLITE_NOMEM ){ sqlite3_result_error_nomem(ctx); jsonStringReset(p); return; } } sqlite3_result_text64(p->pCtx, sqlite3RCStrRef(p->zBuf), p->nUsed, sqlite3RCStrUnref, SQLITE_UTF8); }else{ sqlite3_result_error_nomem(p->pCtx); } }else if( p->eErr & JSTRING_OOM ){ sqlite3_result_error_nomem(p->pCtx); }else if( p->eErr & JSTRING_MALFORMED ){ sqlite3_result_error(p->pCtx, "malformed JSON", -1); } jsonStringReset(p); } /************************************************************************** ** Utility routines for dealing with JsonParse objects **************************************************************************/ /* ** Reclaim all memory allocated by a JsonParse object. But do not ** delete the JsonParse object itself. */ static void jsonParseReset(JsonParse *pParse){ assert( pParse->nJPRef<=1 ); if( pParse->bJsonIsRCStr ){ sqlite3RCStrUnref(pParse->zJson); pParse->zJson = 0; pParse->nJson = 0; pParse->bJsonIsRCStr = 0; } if( pParse->nBlobAlloc ){ sqlite3DbFree(pParse->db, pParse->aBlob); pParse->aBlob = 0; pParse->nBlob = 0; pParse->nBlobAlloc = 0; } } /* ** Decrement the reference count on the JsonParse object. When the ** count reaches zero, free the object. */ static void jsonParseFree(JsonParse *pParse){ if( pParse ){ if( pParse->nJPRef>1 ){ pParse->nJPRef--; }else{ jsonParseReset(pParse); sqlite3DbFree(pParse->db, pParse); } } } /************************************************************************** ** Utility routines for the JSON text parser **************************************************************************/ /* ** Translate a single byte of Hex into an integer. ** This routine only gives a correct answer if h really is a valid hexadecimal ** character: 0..9a..fA..F. But unlike sqlite3HexToInt(), it does not ** assert() if the digit is not hex. */ static u8 jsonHexToInt(int h){ #ifdef SQLITE_ASCII h += 9*(1&(h>>6)); #endif #ifdef SQLITE_EBCDIC h += 9*(1&~(h>>4)); #endif return (u8)(h & 0xf); } /* ** Convert a 4-byte hex string into an integer */ static u32 jsonHexToInt4(const char *z){ u32 v; v = (jsonHexToInt(z[0])<<12) + (jsonHexToInt(z[1])<<8) + (jsonHexToInt(z[2])<<4) + jsonHexToInt(z[3]); return v; } /* ** Return true if z[] begins with 2 (or more) hexadecimal digits */ static int jsonIs2Hex(const char *z){ return sqlite3Isxdigit(z[0]) && sqlite3Isxdigit(z[1]); } |
︙ | ︙ | |||
203832 203833 203834 203835 203836 203837 203838 | char c2; char n; char eType; char nRepl; char *zMatch; char *zRepl; } aNanInfName[] = { | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < > > > > | | | | | | | | | > | < > | | > | > | | > | | | < | > | | | | | | | > | > | < | | | | | | | | | | > | > > > > > > > > > | | | > > > | | | | | | < | < | | | | | | > | | | | | > > > > | | 204197 204198 204199 204200 204201 204202 204203 204204 204205 204206 204207 204208 204209 204210 204211 204212 204213 204214 204215 204216 204217 204218 204219 204220 204221 204222 204223 204224 204225 204226 204227 204228 204229 204230 204231 204232 204233 204234 204235 204236 204237 204238 204239 204240 204241 204242 204243 204244 204245 204246 204247 204248 204249 204250 204251 204252 204253 204254 204255 204256 204257 204258 204259 204260 204261 204262 204263 204264 204265 204266 204267 204268 204269 204270 204271 204272 204273 204274 204275 204276 204277 204278 204279 204280 204281 204282 204283 204284 204285 204286 204287 204288 204289 204290 204291 204292 204293 204294 204295 204296 204297 204298 204299 204300 204301 204302 204303 204304 204305 204306 204307 204308 204309 204310 204311 204312 204313 204314 204315 204316 204317 204318 204319 204320 204321 204322 204323 204324 204325 204326 204327 204328 204329 204330 204331 204332 204333 204334 204335 204336 204337 204338 204339 204340 204341 204342 204343 204344 204345 204346 204347 204348 204349 204350 204351 204352 204353 204354 204355 204356 204357 204358 204359 204360 204361 204362 204363 204364 204365 204366 204367 204368 204369 204370 204371 204372 204373 204374 204375 204376 204377 204378 204379 204380 204381 204382 204383 204384 204385 204386 204387 204388 204389 204390 204391 204392 204393 204394 204395 204396 204397 204398 204399 204400 204401 204402 204403 204404 204405 204406 204407 204408 204409 204410 204411 204412 204413 204414 204415 204416 204417 204418 204419 204420 204421 204422 204423 204424 204425 204426 204427 204428 204429 204430 204431 204432 204433 204434 204435 204436 204437 204438 204439 204440 204441 204442 204443 204444 204445 204446 204447 204448 204449 204450 204451 204452 204453 204454 204455 204456 204457 204458 204459 204460 204461 204462 204463 204464 204465 204466 204467 204468 204469 204470 204471 204472 204473 204474 204475 204476 204477 204478 204479 204480 204481 204482 204483 204484 204485 204486 204487 204488 204489 204490 204491 204492 204493 204494 204495 204496 204497 204498 204499 204500 204501 204502 204503 204504 204505 204506 204507 204508 204509 204510 204511 204512 204513 204514 204515 204516 204517 204518 204519 204520 204521 204522 204523 204524 204525 204526 204527 204528 204529 204530 204531 204532 204533 204534 204535 204536 204537 204538 204539 204540 204541 204542 204543 204544 204545 204546 204547 204548 204549 204550 204551 204552 204553 204554 204555 204556 204557 204558 204559 204560 204561 204562 204563 204564 204565 204566 204567 204568 204569 204570 204571 204572 204573 204574 204575 204576 204577 204578 204579 204580 204581 204582 204583 204584 204585 204586 204587 204588 204589 204590 204591 204592 204593 204594 204595 204596 204597 204598 204599 204600 204601 204602 204603 204604 204605 204606 204607 204608 204609 204610 204611 204612 204613 204614 204615 204616 204617 204618 204619 204620 204621 204622 204623 204624 204625 204626 204627 204628 204629 204630 204631 204632 204633 204634 204635 204636 204637 204638 204639 204640 204641 204642 204643 204644 204645 204646 204647 204648 204649 204650 204651 204652 204653 204654 204655 204656 204657 204658 204659 204660 204661 204662 204663 204664 204665 204666 204667 204668 204669 204670 204671 204672 204673 204674 204675 204676 204677 204678 204679 204680 204681 204682 204683 204684 204685 204686 204687 204688 204689 204690 204691 204692 204693 204694 204695 204696 204697 204698 204699 204700 204701 204702 204703 204704 204705 204706 204707 204708 204709 204710 204711 204712 204713 204714 204715 204716 204717 204718 204719 204720 204721 204722 204723 204724 204725 204726 204727 204728 204729 204730 204731 204732 204733 204734 204735 204736 204737 204738 204739 204740 204741 204742 204743 204744 204745 204746 204747 204748 204749 204750 204751 204752 204753 204754 204755 204756 204757 204758 204759 204760 204761 204762 204763 204764 204765 204766 204767 204768 204769 204770 204771 204772 204773 204774 204775 204776 204777 204778 204779 204780 204781 204782 204783 204784 204785 204786 204787 204788 204789 204790 204791 204792 204793 204794 204795 204796 204797 204798 204799 204800 204801 204802 204803 204804 204805 204806 204807 204808 204809 204810 204811 204812 204813 204814 204815 204816 204817 204818 204819 204820 204821 204822 204823 204824 204825 204826 204827 204828 204829 204830 204831 204832 204833 204834 204835 204836 204837 204838 204839 204840 204841 204842 204843 204844 204845 204846 204847 204848 204849 204850 204851 204852 204853 204854 204855 204856 204857 204858 204859 204860 204861 204862 204863 204864 204865 204866 204867 204868 204869 204870 204871 204872 204873 204874 204875 204876 204877 204878 204879 204880 204881 204882 204883 204884 204885 204886 204887 204888 204889 204890 204891 204892 204893 204894 204895 204896 204897 204898 204899 204900 204901 204902 204903 204904 204905 204906 204907 204908 204909 204910 204911 204912 204913 204914 204915 204916 204917 204918 204919 204920 204921 204922 204923 204924 204925 204926 204927 204928 204929 204930 204931 204932 204933 204934 204935 204936 204937 204938 204939 204940 204941 204942 204943 204944 204945 204946 204947 204948 204949 204950 204951 204952 204953 204954 204955 204956 204957 204958 204959 204960 204961 204962 204963 204964 204965 204966 204967 204968 204969 204970 204971 204972 204973 204974 204975 204976 204977 204978 204979 204980 204981 204982 204983 204984 204985 204986 204987 204988 204989 204990 204991 204992 204993 204994 204995 204996 204997 204998 204999 205000 205001 205002 205003 205004 205005 205006 205007 205008 205009 205010 205011 205012 205013 205014 205015 205016 205017 205018 205019 205020 205021 205022 205023 205024 205025 205026 205027 205028 205029 205030 205031 205032 205033 205034 205035 205036 205037 205038 205039 | char c2; char n; char eType; char nRepl; char *zMatch; char *zRepl; } aNanInfName[] = { { 'i', 'I', 3, JSONB_FLOAT, 7, "inf", "9.0e999" }, { 'i', 'I', 8, JSONB_FLOAT, 7, "infinity", "9.0e999" }, { 'n', 'N', 3, JSONB_NULL, 4, "NaN", "null" }, { 'q', 'Q', 4, JSONB_NULL, 4, "QNaN", "null" }, { 's', 'S', 4, JSONB_NULL, 4, "SNaN", "null" }, }; /* ** Report the wrong number of arguments for json_insert(), json_replace() ** or json_set(). */ static void jsonWrongNumArgs( sqlite3_context *pCtx, const char *zFuncName ){ char *zMsg = sqlite3_mprintf("json_%s() needs an odd number of arguments", zFuncName); sqlite3_result_error(pCtx, zMsg, -1); sqlite3_free(zMsg); } /**************************************************************************** ** Utility routines for dealing with the binary BLOB representation of JSON ****************************************************************************/ /* ** Expand pParse->aBlob so that it holds at least N bytes. ** ** Return the number of errors. */ static int jsonBlobExpand(JsonParse *pParse, u32 N){ u8 *aNew; u32 t; assert( N>pParse->nBlobAlloc ); if( pParse->nBlobAlloc==0 ){ t = 100; }else{ t = pParse->nBlobAlloc*2; } if( t<N ) t = N+100; aNew = sqlite3DbRealloc(pParse->db, pParse->aBlob, t); if( aNew==0 ){ pParse->oom = 1; return 1; } pParse->aBlob = aNew; pParse->nBlobAlloc = t; return 0; } /* ** If pParse->aBlob is not previously editable (because it is taken ** from sqlite3_value_blob(), as indicated by the fact that ** pParse->nBlobAlloc==0 and pParse->nBlob>0) then make it editable ** by making a copy into space obtained from malloc. ** ** Return true on success. Return false on OOM. */ static int jsonBlobMakeEditable(JsonParse *pParse, u32 nExtra){ u8 *aOld; u32 nSize; assert( !pParse->bReadOnly ); if( pParse->oom ) return 0; if( pParse->nBlobAlloc>0 ) return 1; aOld = pParse->aBlob; nSize = pParse->nBlob + nExtra; pParse->aBlob = 0; if( jsonBlobExpand(pParse, nSize) ){ return 0; } assert( pParse->nBlobAlloc >= pParse->nBlob + nExtra ); memcpy(pParse->aBlob, aOld, pParse->nBlob); return 1; } /* Expand pParse->aBlob and append one bytes. */ static SQLITE_NOINLINE void jsonBlobExpandAndAppendOneByte( JsonParse *pParse, u8 c ){ jsonBlobExpand(pParse, pParse->nBlob+1); if( pParse->oom==0 ){ assert( pParse->nBlob+1<=pParse->nBlobAlloc ); pParse->aBlob[pParse->nBlob++] = c; } } /* Append a single character. */ static void jsonBlobAppendOneByte(JsonParse *pParse, u8 c){ if( pParse->nBlob >= pParse->nBlobAlloc ){ jsonBlobExpandAndAppendOneByte(pParse, c); }else{ pParse->aBlob[pParse->nBlob++] = c; } } /* Slow version of jsonBlobAppendNode() that first resizes the ** pParse->aBlob structure. */ static void jsonBlobAppendNode(JsonParse*,u8,u32,const void*); static SQLITE_NOINLINE void jsonBlobExpandAndAppendNode( JsonParse *pParse, u8 eType, u32 szPayload, const void *aPayload ){ if( jsonBlobExpand(pParse, pParse->nBlob+szPayload+9) ) return; jsonBlobAppendNode(pParse, eType, szPayload, aPayload); } /* Append an node type byte together with the payload size and ** possibly also the payload. ** ** If aPayload is not NULL, then it is a pointer to the payload which ** is also appended. If aPayload is NULL, the pParse->aBlob[] array ** is resized (if necessary) so that it is big enough to hold the ** payload, but the payload is not appended and pParse->nBlob is left ** pointing to where the first byte of payload will eventually be. */ static void jsonBlobAppendNode( JsonParse *pParse, /* The JsonParse object under construction */ u8 eType, /* Node type. One of JSONB_* */ u32 szPayload, /* Number of bytes of payload */ const void *aPayload /* The payload. Might be NULL */ ){ u8 *a; if( pParse->nBlob+szPayload+9 > pParse->nBlobAlloc ){ jsonBlobExpandAndAppendNode(pParse,eType,szPayload,aPayload); return; } assert( pParse->aBlob!=0 ); a = &pParse->aBlob[pParse->nBlob]; if( szPayload<=11 ){ a[0] = eType | (szPayload<<4); pParse->nBlob += 1; }else if( szPayload<=0xff ){ a[0] = eType | 0xc0; a[1] = szPayload & 0xff; pParse->nBlob += 2; }else if( szPayload<=0xffff ){ a[0] = eType | 0xd0; a[1] = (szPayload >> 8) & 0xff; a[2] = szPayload & 0xff; pParse->nBlob += 3; }else{ a[0] = eType | 0xe0; a[1] = (szPayload >> 24) & 0xff; a[2] = (szPayload >> 16) & 0xff; a[3] = (szPayload >> 8) & 0xff; a[4] = szPayload & 0xff; pParse->nBlob += 5; } if( aPayload ){ pParse->nBlob += szPayload; memcpy(&pParse->aBlob[pParse->nBlob-szPayload], aPayload, szPayload); } } /* Change the payload size for the node at index i to be szPayload. */ static int jsonBlobChangePayloadSize( JsonParse *pParse, u32 i, u32 szPayload ){ u8 *a; u8 szType; u8 nExtra; u8 nNeeded; int delta; if( pParse->oom ) return 0; a = &pParse->aBlob[i]; szType = a[0]>>4; if( szType<=11 ){ nExtra = 0; }else if( szType==12 ){ nExtra = 1; }else if( szType==13 ){ nExtra = 2; }else{ nExtra = 4; } if( szPayload<=11 ){ nNeeded = 0; }else if( szPayload<=0xff ){ nNeeded = 1; }else if( szPayload<=0xffff ){ nNeeded = 2; }else{ nNeeded = 4; } delta = nNeeded - nExtra; if( delta ){ u32 newSize = pParse->nBlob + delta; if( delta>0 ){ if( newSize>pParse->nBlobAlloc && jsonBlobExpand(pParse, newSize) ){ return 0; /* OOM error. Error state recorded in pParse->oom. */ } a = &pParse->aBlob[i]; memmove(&a[1+delta], &a[1], pParse->nBlob - (i+1)); }else{ memmove(&a[1], &a[1-delta], pParse->nBlob - (i+1-delta)); } pParse->nBlob = newSize; } if( nNeeded==0 ){ a[0] = (a[0] & 0x0f) | (szPayload<<4); }else if( nNeeded==1 ){ a[0] = (a[0] & 0x0f) | 0xc0; a[1] = szPayload & 0xff; }else if( nNeeded==2 ){ a[0] = (a[0] & 0x0f) | 0xd0; a[1] = (szPayload >> 8) & 0xff; a[2] = szPayload & 0xff; }else{ a[0] = (a[0] & 0x0f) | 0xe0; a[1] = (szPayload >> 24) & 0xff; a[2] = (szPayload >> 16) & 0xff; a[3] = (szPayload >> 8) & 0xff; a[4] = szPayload & 0xff; } return delta; } /* ** If z[0] is 'u' and is followed by exactly 4 hexadecimal character, ** then set *pOp to JSONB_TEXTJ and return true. If not, do not make ** any changes to *pOp and return false. */ static int jsonIs4HexB(const char *z, int *pOp){ if( z[0]!='u' ) return 0; if( !jsonIs4Hex(&z[1]) ) return 0; *pOp = JSONB_TEXTJ; return 1; } /* ** Check a single element of the JSONB in pParse for validity. ** ** The element to be checked starts at offset i and must end at on the ** last byte before iEnd. ** ** Return 0 if everything is correct. Return the 1-based byte offset of the ** error if a problem is detected. (In other words, if the error is at offset ** 0, return 1). */ static u32 jsonbValidityCheck( const JsonParse *pParse, /* Input JSONB. Only aBlob and nBlob are used */ u32 i, /* Start of element as pParse->aBlob[i] */ u32 iEnd, /* One more than the last byte of the element */ u32 iDepth /* Current nesting depth */ ){ u32 n, sz, j, k; const u8 *z; u8 x; if( iDepth>JSON_MAX_DEPTH ) return i+1; sz = 0; n = jsonbPayloadSize(pParse, i, &sz); if( NEVER(n==0) ) return i+1; /* Checked by caller */ if( NEVER(i+n+sz!=iEnd) ) return i+1; /* Checked by caller */ z = pParse->aBlob; x = z[i] & 0x0f; switch( x ){ case JSONB_NULL: case JSONB_TRUE: case JSONB_FALSE: { return n+sz==1 ? 0 : i+1; } case JSONB_INT: { if( sz<1 ) return i+1; j = i+n; if( z[j]=='-' ){ j++; if( sz<2 ) return i+1; } k = i+n+sz; while( j<k ){ if( sqlite3Isdigit(z[j]) ){ j++; }else{ return j+1; } } return 0; } case JSONB_INT5: { if( sz<3 ) return i+1; j = i+n; if( z[j]=='-' ){ if( sz<4 ) return i+1; j++; } if( z[j]!='0' ) return i+1; if( z[j+1]!='x' && z[j+1]!='X' ) return j+2; j += 2; k = i+n+sz; while( j<k ){ if( sqlite3Isxdigit(z[j]) ){ j++; }else{ return j+1; } } return 0; } case JSONB_FLOAT: case JSONB_FLOAT5: { u8 seen = 0; /* 0: initial. 1: '.' seen 2: 'e' seen */ if( sz<2 ) return i+1; j = i+n; k = j+sz; if( z[j]=='-' ){ j++; if( sz<3 ) return i+1; } if( z[j]=='.' ){ if( x==JSONB_FLOAT ) return j+1; if( !sqlite3Isdigit(z[j+1]) ) return j+1; j += 2; seen = 1; }else if( z[j]=='0' && x==JSONB_FLOAT ){ if( j+3>k ) return j+1; if( z[j+1]!='.' && z[j+1]!='e' && z[j+1]!='E' ) return j+1; j++; } for(; j<k; j++){ if( sqlite3Isdigit(z[j]) ) continue; if( z[j]=='.' ){ if( seen>0 ) return j+1; if( x==JSONB_FLOAT && (j==k-1 || !sqlite3Isdigit(z[j+1])) ){ return j+1; } seen = 1; continue; } if( z[j]=='e' || z[j]=='E' ){ if( seen==2 ) return j+1; if( j==k-1 ) return j+1; if( z[j+1]=='+' || z[j+1]=='-' ){ j++; if( j==k-1 ) return j+1; } seen = 2; continue; } return j+1; } if( seen==0 ) return i+1; return 0; } case JSONB_TEXT: { j = i+n; k = j+sz; while( j<k ){ if( !jsonIsOk[z[j]] && z[j]!='\'' ) return j+1; j++; } return 0; } case JSONB_TEXTJ: case JSONB_TEXT5: { j = i+n; k = j+sz; while( j<k ){ if( !jsonIsOk[z[j]] && z[j]!='\'' ){ if( z[j]=='"' ){ if( x==JSONB_TEXTJ ) return j+1; }else if( z[j]!='\\' || j+1>=k ){ return j+1; }else if( strchr("\"\\/bfnrt",z[j+1])!=0 ){ j++; }else if( z[j+1]=='u' ){ if( j+5>=k ) return j+1; if( !jsonIs4Hex((const char*)&z[j+2]) ) return j+1; j++; }else if( x!=JSONB_TEXT5 ){ return j+1; }else{ u32 c = 0; u32 szC = jsonUnescapeOneChar((const char*)&z[j], k-j, &c); if( c==JSON_INVALID_CHAR ) return j+1; j += szC - 1; } } j++; } return 0; } case JSONB_TEXTRAW: { return 0; } case JSONB_ARRAY: { u32 sub; j = i+n; k = j+sz; while( j<k ){ sz = 0; n = jsonbPayloadSize(pParse, j, &sz); if( n==0 ) return j+1; if( j+n+sz>k ) return j+1; sub = jsonbValidityCheck(pParse, j, j+n+sz, iDepth+1); if( sub ) return sub; j += n + sz; } assert( j==k ); return 0; } case JSONB_OBJECT: { u32 cnt = 0; u32 sub; j = i+n; k = j+sz; while( j<k ){ sz = 0; n = jsonbPayloadSize(pParse, j, &sz); if( n==0 ) return j+1; if( j+n+sz>k ) return j+1; if( (cnt & 1)==0 ){ x = z[j] & 0x0f; if( x<JSONB_TEXT || x>JSONB_TEXTRAW ) return j+1; } sub = jsonbValidityCheck(pParse, j, j+n+sz, iDepth+1); if( sub ) return sub; cnt++; j += n + sz; } assert( j==k ); if( (cnt & 1)!=0 ) return j+1; return 0; } default: { return i+1; } } } /* ** Translate a single element of JSON text at pParse->zJson[i] into ** its equivalent binary JSONB representation. Append the translation into ** pParse->aBlob[] beginning at pParse->nBlob. The size of ** pParse->aBlob[] is increased as necessary. ** ** Return the index of the first character past the end of the element parsed, ** or one of the following special result codes: ** ** 0 End of input ** -1 Syntax error or OOM ** -2 '}' seen \ ** -3 ']' seen \___ For these returns, pParse->iErr is set to ** -4 ',' seen / the index in zJson[] of the seen character ** -5 ':' seen / */ static int jsonTranslateTextToBlob(JsonParse *pParse, u32 i){ char c; u32 j; u32 iThis, iStart; int x; u8 t; const char *z = pParse->zJson; json_parse_restart: switch( (u8)z[i] ){ case '{': { /* Parse object */ iThis = pParse->nBlob; jsonBlobAppendNode(pParse, JSONB_OBJECT, pParse->nJson-i, 0); if( ++pParse->iDepth > JSON_MAX_DEPTH ){ pParse->iErr = i; return -1; } iStart = pParse->nBlob; for(j=i+1;;j++){ u32 iBlob = pParse->nBlob; x = jsonTranslateTextToBlob(pParse, j); if( x<=0 ){ int op; if( x==(-2) ){ j = pParse->iErr; if( pParse->nBlob!=(u32)iStart ) pParse->hasNonstd = 1; break; } j += json5Whitespace(&z[j]); op = JSONB_TEXT; if( sqlite3JsonId1(z[j]) || (z[j]=='\\' && jsonIs4HexB(&z[j+1], &op)) ){ int k = j+1; while( (sqlite3JsonId2(z[k]) && json5Whitespace(&z[k])==0) || (z[k]=='\\' && jsonIs4HexB(&z[k+1], &op)) ){ k++; } assert( iBlob==pParse->nBlob ); jsonBlobAppendNode(pParse, op, k-j, &z[j]); pParse->hasNonstd = 1; x = k; }else{ if( x!=-1 ) pParse->iErr = j; return -1; } } if( pParse->oom ) return -1; t = pParse->aBlob[iBlob] & 0x0f; if( t<JSONB_TEXT || t>JSONB_TEXTRAW ){ pParse->iErr = j; return -1; } j = x; if( z[j]==':' ){ j++; }else{ if( jsonIsspace(z[j]) ){ /* strspn() is not helpful here */ do{ j++; }while( jsonIsspace(z[j]) ); if( z[j]==':' ){ j++; goto parse_object_value; } } x = jsonTranslateTextToBlob(pParse, j); if( x!=(-5) ){ if( x!=(-1) ) pParse->iErr = j; return -1; } j = pParse->iErr+1; } parse_object_value: x = jsonTranslateTextToBlob(pParse, j); if( x<=0 ){ if( x!=(-1) ) pParse->iErr = j; return -1; } j = x; if( z[j]==',' ){ continue; }else if( z[j]=='}' ){ break; }else{ if( jsonIsspace(z[j]) ){ j += 1 + (u32)strspn(&z[j+1], jsonSpaces); if( z[j]==',' ){ continue; }else if( z[j]=='}' ){ break; } } x = jsonTranslateTextToBlob(pParse, j); if( x==(-4) ){ j = pParse->iErr; continue; } if( x==(-2) ){ j = pParse->iErr; break; } } pParse->iErr = j; return -1; } jsonBlobChangePayloadSize(pParse, iThis, pParse->nBlob - iStart); pParse->iDepth--; return j+1; } case '[': { /* Parse array */ iThis = pParse->nBlob; jsonBlobAppendNode(pParse, JSONB_ARRAY, pParse->nJson - i, 0); iStart = pParse->nBlob; if( pParse->oom ) return -1; if( ++pParse->iDepth > JSON_MAX_DEPTH ){ pParse->iErr = i; return -1; } for(j=i+1;;j++){ x = jsonTranslateTextToBlob(pParse, j); if( x<=0 ){ if( x==(-3) ){ j = pParse->iErr; if( pParse->nBlob!=iStart ) pParse->hasNonstd = 1; break; } if( x!=(-1) ) pParse->iErr = j; return -1; } j = x; if( z[j]==',' ){ continue; }else if( z[j]==']' ){ break; }else{ if( jsonIsspace(z[j]) ){ j += 1 + (u32)strspn(&z[j+1], jsonSpaces); if( z[j]==',' ){ continue; }else if( z[j]==']' ){ break; } } x = jsonTranslateTextToBlob(pParse, j); if( x==(-4) ){ j = pParse->iErr; continue; } if( x==(-3) ){ j = pParse->iErr; break; } } pParse->iErr = j; return -1; } jsonBlobChangePayloadSize(pParse, iThis, pParse->nBlob - iStart); pParse->iDepth--; return j+1; } case '\'': { u8 opcode; char cDelim; pParse->hasNonstd = 1; opcode = JSONB_TEXT; goto parse_string; case '"': /* Parse string */ opcode = JSONB_TEXT; parse_string: cDelim = z[i]; j = i+1; while( 1 /*exit-by-break*/ ){ if( jsonIsOk[(u8)z[j]] ){ if( !jsonIsOk[(u8)z[j+1]] ){ j += 1; }else if( !jsonIsOk[(u8)z[j+2]] ){ j += 2; }else{ j += 3; continue; } } c = z[j]; if( c==cDelim ){ break; }else if( c=='\\' ){ c = z[++j]; if( c=='"' || c=='\\' || c=='/' || c=='b' || c=='f' || c=='n' || c=='r' || c=='t' || (c=='u' && jsonIs4Hex(&z[j+1])) ){ if( opcode==JSONB_TEXT ) opcode = JSONB_TEXTJ; }else if( c=='\'' || c=='0' || c=='v' || c=='\n' || (0xe2==(u8)c && 0x80==(u8)z[j+1] && (0xa8==(u8)z[j+2] || 0xa9==(u8)z[j+2])) || (c=='x' && jsonIs2Hex(&z[j+1])) ){ opcode = JSONB_TEXT5; pParse->hasNonstd = 1; }else if( c=='\r' ){ if( z[j+1]=='\n' ) j++; opcode = JSONB_TEXT5; pParse->hasNonstd = 1; }else{ pParse->iErr = j; return -1; } }else if( c<=0x1f ){ /* Control characters are not allowed in strings */ pParse->iErr = j; return -1; }else if( c=='"' ){ opcode = JSONB_TEXT5; } j++; } jsonBlobAppendNode(pParse, opcode, j-1-i, &z[i+1]); return j+1; } case 't': { if( strncmp(z+i,"true",4)==0 && !sqlite3Isalnum(z[i+4]) ){ jsonBlobAppendOneByte(pParse, JSONB_TRUE); return i+4; } pParse->iErr = i; return -1; } case 'f': { if( strncmp(z+i,"false",5)==0 && !sqlite3Isalnum(z[i+5]) ){ jsonBlobAppendOneByte(pParse, JSONB_FALSE); return i+5; } pParse->iErr = i; return -1; } case '+': { u8 seenE; pParse->hasNonstd = 1; t = 0x00; /* Bit 0x01: JSON5. Bit 0x02: FLOAT */ goto parse_number; case '.': if( sqlite3Isdigit(z[i+1]) ){ pParse->hasNonstd = 1; t = 0x03; /* Bit 0x01: JSON5. Bit 0x02: FLOAT */ seenE = 0; goto parse_number_2; } pParse->iErr = i; return -1; case '-': case '0': case '1': case '2': case '3': case '4': case '5': case '6': case '7': case '8': case '9': /* Parse number */ t = 0x00; /* Bit 0x01: JSON5. Bit 0x02: FLOAT */ parse_number: seenE = 0; assert( '-' < '0' ); assert( '+' < '0' ); assert( '.' < '0' ); c = z[i]; if( c<='0' ){ if( c=='0' ){ if( (z[i+1]=='x' || z[i+1]=='X') && sqlite3Isxdigit(z[i+2]) ){ assert( t==0x00 ); pParse->hasNonstd = 1; t = 0x01; for(j=i+3; sqlite3Isxdigit(z[j]); j++){} goto parse_number_finish; }else if( sqlite3Isdigit(z[i+1]) ){ pParse->iErr = i+1; return -1; } }else{ if( !sqlite3Isdigit(z[i+1]) ){ /* JSON5 allows for "+Infinity" and "-Infinity" using exactly ** that case. SQLite also allows these in any case and it allows ** "+inf" and "-inf". */ if( (z[i+1]=='I' || z[i+1]=='i') && sqlite3StrNICmp(&z[i+1], "inf",3)==0 ){ pParse->hasNonstd = 1; if( z[i]=='-' ){ jsonBlobAppendNode(pParse, JSONB_FLOAT, 6, "-9e999"); }else{ jsonBlobAppendNode(pParse, JSONB_FLOAT, 5, "9e999"); } return i + (sqlite3StrNICmp(&z[i+4],"inity",5)==0 ? 9 : 4); } if( z[i+1]=='.' ){ pParse->hasNonstd = 1; t |= 0x01; goto parse_number_2; } pParse->iErr = i; return -1; } if( z[i+1]=='0' ){ if( sqlite3Isdigit(z[i+2]) ){ pParse->iErr = i+1; return -1; }else if( (z[i+2]=='x' || z[i+2]=='X') && sqlite3Isxdigit(z[i+3]) ){ pParse->hasNonstd = 1; t |= 0x01; for(j=i+4; sqlite3Isxdigit(z[j]); j++){} goto parse_number_finish; } } } } parse_number_2: for(j=i+1;; j++){ c = z[j]; if( sqlite3Isdigit(c) ) continue; if( c=='.' ){ if( (t & 0x02)!=0 ){ pParse->iErr = j; return -1; } t |= 0x02; continue; } if( c=='e' || c=='E' ){ if( z[j-1]<'0' ){ if( ALWAYS(z[j-1]=='.') && ALWAYS(j-2>=i) && sqlite3Isdigit(z[j-2]) ){ pParse->hasNonstd = 1; t |= 0x01; }else{ pParse->iErr = j; return -1; } } if( seenE ){ pParse->iErr = j; return -1; } t |= 0x02; seenE = 1; c = z[j+1]; if( c=='+' || c=='-' ){ j++; c = z[j+1]; } if( c<'0' || c>'9' ){ pParse->iErr = j; return -1; } continue; } break; } if( z[j-1]<'0' ){ if( ALWAYS(z[j-1]=='.') && ALWAYS(j-2>=i) && sqlite3Isdigit(z[j-2]) ){ pParse->hasNonstd = 1; t |= 0x01; }else{ pParse->iErr = j; return -1; } } parse_number_finish: assert( JSONB_INT+0x01==JSONB_INT5 ); assert( JSONB_FLOAT+0x01==JSONB_FLOAT5 ); assert( JSONB_INT+0x02==JSONB_FLOAT ); if( z[i]=='+' ) i++; jsonBlobAppendNode(pParse, JSONB_INT+t, j-i, &z[i]); return j; } case '}': { pParse->iErr = i; return -2; /* End of {...} */ } case ']': { |
︙ | ︙ | |||
204232 204233 204234 204235 204236 204237 204238 | case 0: { return 0; /* End of file */ } case 0x09: case 0x0a: case 0x0d: case 0x20: { | | < < | 205051 205052 205053 205054 205055 205056 205057 205058 205059 205060 205061 205062 205063 205064 205065 | case 0: { return 0; /* End of file */ } case 0x09: case 0x0a: case 0x0d: case 0x20: { i += 1 + (u32)strspn(&z[i+1], jsonSpaces); goto json_parse_restart; } case 0x0b: case 0x0c: case '/': case 0xc2: case 0xe1: |
︙ | ︙ | |||
204256 204257 204258 204259 204260 204261 204262 | goto json_parse_restart; } pParse->iErr = i; return -1; } case 'n': { if( strncmp(z+i,"null",4)==0 && !sqlite3Isalnum(z[i+4]) ){ | | | | > > > > | | > > > > > | > > > > > > > > > > > > > > > > > > > > > | > | < > > > > | < > | > | < < < < < > | | > > > > | | > > > | > > | > > | > > > > > | > > | > > > > > > > > | > > > > > > > > > > > > > | | | < | > > > > > > > > > > > | > > > | | | | > | > > > > | | > | | | > > > > > > > > | < > > | < > > > > > > > > > > > > > > > > > > > > > | < > > > > > > > > > > > > > | < > > > | < < < | > | | > > > | < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < < | > > | < > | | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > | < < > > | | < < < < > > | > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > | > > > > > > > > > | < > > | | > > > > > > > > > > > | > > > > | | | < > | < > > | | > | > > > > | > > | > | < > > > > > | | > > > | > > > > > > > > > | < > > > > > | | | | | | > > | < > | > > > > | < > > > > > > > > > > > | > > > | > > > > | | | > | | > > > > | > > > > | > > > > > > > | > > > > > | > > | > > > | < | | > > > > > > > > > > > > | < < > > > > > > > > > > | < < > > | > | > > | | | > > > | | > > | > > > | > | < | > | < < | < | < < > | | > > | > | > > | > > > > > > > > > | > > > > > > > > > > > | > > > > > | > > > > > > > > > > > > > > > > > > > > > > > | | | < > > > > > > > | > > | | | | | < > | > > > > > | > | < < | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | < | > | < > | > > > > > > > | < > > | | | | < | > | | | < < | | < < < | < | | | < > | > > > > < > | | < < > | < | > < | > > > | < | > > > > > > > > > > | > > > > > | > > | | > | < < | < < < | > | | > | | | > | > | | > > > | > > | | | > | < > | | > | > > > | > > > | | | | | | > | > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > | | > | > > > > | > > | | | > > > > > > | > > > > > > > > > > | > > > > > | | > > > | < < < | | < < | | < | > | < > | | | > > | | > > > > > > > > < > | < | < > | < > > > | < | > > > | > > > > > > > > > > > > > | > > > > > | > > > > | > > | | > > > | > > > > > > > > > > > > > > > > > > > > > > | | < > > | | < < | < < | > > > > > > | | | < < < > > > > > > > | | > > | < | > > | > > | < > > > > > > > > > > > > | > > | < < < > > > > > > > | > > > > > > > > > > > > > > > | > > > > > > | | | < > | > > > > > | | | > > | > > > > > > | < < | > | < < > | < | | > > | > > > > > > | > > < < > | > > > > > > > > > > > > > > > > | > > > > < < > > | < < < | > > > > > | > | | > > > > > > > | > > > > | > > > > > | > > > > > > > > > | | > > > > > | > | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > | > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > > | | > > > > > > | > > > > > > > > > > > > > | | > > > > > | > | | > < < | < < < < | < > > | > | | | > > > > > > | > > > | | > > > | < | < < < < | < < | | > > > > > > > | > > > > > > > | | > | > | < < > > > > > > > | > > > > > > > > > > > | < < < < < > < | > | | < < < < < > > > > > | | < < < < | | | | | > > > > > > > > > > > > > > > > > | > > > > > | < < < > > > > | < < | > > > | > > > > > > > > > > > > > > > > > > > | > | > > > > > | > > > | | | < > | > | < < | < < < < < < < < < | < > | > | < < < < < | < < < < < < | | | | | | | < | | | < | | | | > | > > > > | > | < < | < < | | < < | | | > > | > | < | < < | | > > | > > > > > > | 205073 205074 205075 205076 205077 205078 205079 205080 205081 205082 205083 205084 205085 205086 205087 205088 205089 205090 205091 205092 205093 205094 205095 205096 205097 205098 205099 205100 205101 205102 205103 205104 205105 205106 205107 205108 205109 205110 205111 205112 205113 205114 205115 205116 205117 205118 205119 205120 205121 205122 205123 205124 205125 205126 205127 205128 205129 205130 205131 205132 205133 205134 205135 205136 205137 205138 205139 205140 205141 205142 205143 205144 205145 205146 205147 205148 205149 205150 205151 205152 205153 205154 205155 205156 205157 205158 205159 205160 205161 205162 205163 205164 205165 205166 205167 205168 205169 205170 205171 205172 205173 205174 205175 205176 205177 205178 205179 205180 205181 205182 205183 205184 205185 205186 205187 205188 205189 205190 205191 205192 205193 205194 205195 205196 205197 205198 205199 205200 205201 205202 205203 205204 205205 205206 205207 205208 205209 205210 205211 205212 205213 205214 205215 205216 205217 205218 205219 205220 205221 205222 205223 205224 205225 205226 205227 205228 205229 205230 205231 205232 205233 205234 205235 205236 205237 205238 205239 205240 205241 205242 205243 205244 205245 205246 205247 205248 205249 205250 205251 205252 205253 205254 205255 205256 205257 205258 205259 205260 205261 205262 205263 205264 205265 205266 205267 205268 205269 205270 205271 205272 205273 205274 205275 205276 205277 205278 205279 205280 205281 205282 205283 205284 205285 205286 205287 205288 205289 205290 205291 205292 205293 205294 205295 205296 205297 205298 205299 205300 205301 205302 205303 205304 205305 205306 205307 205308 205309 205310 205311 205312 205313 205314 205315 205316 205317 205318 205319 205320 205321 205322 205323 205324 205325 205326 205327 205328 205329 205330 205331 205332 205333 205334 205335 205336 205337 205338 205339 205340 205341 205342 205343 205344 205345 205346 205347 205348 205349 205350 205351 205352 205353 205354 205355 205356 205357 205358 205359 205360 205361 205362 205363 205364 205365 205366 205367 205368 205369 205370 205371 205372 205373 205374 205375 205376 205377 205378 205379 205380 205381 205382 205383 205384 205385 205386 205387 205388 205389 205390 205391 205392 205393 205394 205395 205396 205397 205398 205399 205400 205401 205402 205403 205404 205405 205406 205407 205408 205409 205410 205411 205412 205413 205414 205415 205416 205417 205418 205419 205420 205421 205422 205423 205424 205425 205426 205427 205428 205429 205430 205431 205432 205433 205434 205435 205436 205437 205438 205439 205440 205441 205442 205443 205444 205445 205446 205447 205448 205449 205450 205451 205452 205453 205454 205455 205456 205457 205458 205459 205460 205461 205462 205463 205464 205465 205466 205467 205468 205469 205470 205471 205472 205473 205474 205475 205476 205477 205478 205479 205480 205481 205482 205483 205484 205485 205486 205487 205488 205489 205490 205491 205492 205493 205494 205495 205496 205497 205498 205499 205500 205501 205502 205503 205504 205505 205506 205507 205508 205509 205510 205511 205512 205513 205514 205515 205516 205517 205518 205519 205520 205521 205522 205523 205524 205525 205526 205527 205528 205529 205530 205531 205532 205533 205534 205535 205536 205537 205538 205539 205540 205541 205542 205543 205544 205545 205546 205547 205548 205549 205550 205551 205552 205553 205554 205555 205556 205557 205558 205559 205560 205561 205562 205563 205564 205565 205566 205567 205568 205569 205570 205571 205572 205573 205574 205575 205576 205577 205578 205579 205580 205581 205582 205583 205584 205585 205586 205587 205588 205589 205590 205591 205592 205593 205594 205595 205596 205597 205598 205599 205600 205601 205602 205603 205604 205605 205606 205607 205608 205609 205610 205611 205612 205613 205614 205615 205616 205617 205618 205619 205620 205621 205622 205623 205624 205625 205626 205627 205628 205629 205630 205631 205632 205633 205634 205635 205636 205637 205638 205639 205640 205641 205642 205643 205644 205645 205646 205647 205648 205649 205650 205651 205652 205653 205654 205655 205656 205657 205658 205659 205660 205661 205662 205663 205664 205665 205666 205667 205668 205669 205670 205671 205672 205673 205674 205675 205676 205677 205678 205679 205680 205681 205682 205683 205684 205685 205686 205687 205688 205689 205690 205691 205692 205693 205694 205695 205696 205697 205698 205699 205700 205701 205702 205703 205704 205705 205706 205707 205708 205709 205710 205711 205712 205713 205714 205715 205716 205717 205718 205719 205720 205721 205722 205723 205724 205725 205726 205727 205728 205729 205730 205731 205732 205733 205734 205735 205736 205737 205738 205739 205740 205741 205742 205743 205744 205745 205746 205747 205748 205749 205750 205751 205752 205753 205754 205755 205756 205757 205758 205759 205760 205761 205762 205763 205764 205765 205766 205767 205768 205769 205770 205771 205772 205773 205774 205775 205776 205777 205778 205779 205780 205781 205782 205783 205784 205785 205786 205787 205788 205789 205790 205791 205792 205793 205794 205795 205796 205797 205798 205799 205800 205801 205802 205803 205804 205805 205806 205807 205808 205809 205810 205811 205812 205813 205814 205815 205816 205817 205818 205819 205820 205821 205822 205823 205824 205825 205826 205827 205828 205829 205830 205831 205832 205833 205834 205835 205836 205837 205838 205839 205840 205841 205842 205843 205844 205845 205846 205847 205848 205849 205850 205851 205852 205853 205854 205855 205856 205857 205858 205859 205860 205861 205862 205863 205864 205865 205866 205867 205868 205869 205870 205871 205872 205873 205874 205875 205876 205877 205878 205879 205880 205881 205882 205883 205884 205885 205886 205887 205888 205889 205890 205891 205892 205893 205894 205895 205896 205897 205898 205899 205900 205901 205902 205903 205904 205905 205906 205907 205908 205909 205910 205911 205912 205913 205914 205915 205916 205917 205918 205919 205920 205921 205922 205923 205924 205925 205926 205927 205928 205929 205930 205931 205932 205933 205934 205935 205936 205937 205938 205939 205940 205941 205942 205943 205944 205945 205946 205947 205948 205949 205950 205951 205952 205953 205954 205955 205956 205957 205958 205959 205960 205961 205962 205963 205964 205965 205966 205967 205968 205969 205970 205971 205972 205973 205974 205975 205976 205977 205978 205979 205980 205981 205982 205983 205984 205985 205986 205987 205988 205989 205990 205991 205992 205993 205994 205995 205996 205997 205998 205999 206000 206001 206002 206003 206004 206005 206006 206007 206008 206009 206010 206011 206012 206013 206014 206015 206016 206017 206018 206019 206020 206021 206022 206023 206024 206025 206026 206027 206028 206029 206030 206031 206032 206033 206034 206035 206036 206037 206038 206039 206040 206041 206042 206043 206044 206045 206046 206047 206048 206049 206050 206051 206052 206053 206054 206055 206056 206057 206058 206059 206060 206061 206062 206063 206064 206065 206066 206067 206068 206069 206070 206071 206072 206073 206074 206075 206076 206077 206078 206079 206080 206081 206082 206083 206084 206085 206086 206087 206088 206089 206090 206091 206092 206093 206094 206095 206096 206097 206098 206099 206100 206101 206102 206103 206104 206105 206106 206107 206108 206109 206110 206111 206112 206113 206114 206115 206116 206117 206118 206119 206120 206121 206122 206123 206124 206125 206126 206127 206128 206129 206130 206131 206132 206133 206134 206135 206136 206137 206138 206139 206140 206141 206142 206143 206144 206145 206146 206147 206148 206149 206150 206151 206152 206153 206154 206155 206156 206157 206158 206159 206160 206161 206162 206163 206164 206165 206166 206167 206168 206169 206170 206171 206172 206173 206174 206175 206176 206177 206178 206179 206180 206181 206182 206183 206184 206185 206186 206187 206188 206189 206190 206191 206192 206193 206194 206195 206196 206197 206198 206199 206200 206201 206202 206203 206204 206205 206206 206207 206208 206209 206210 206211 206212 206213 206214 206215 206216 206217 206218 206219 206220 206221 206222 206223 206224 206225 206226 206227 206228 206229 206230 206231 206232 206233 206234 206235 206236 206237 206238 206239 206240 206241 206242 206243 206244 206245 206246 206247 206248 206249 206250 206251 206252 206253 206254 206255 206256 206257 206258 206259 206260 206261 206262 206263 206264 206265 206266 206267 206268 206269 206270 206271 206272 206273 206274 206275 206276 206277 206278 206279 206280 206281 206282 206283 206284 206285 206286 206287 206288 206289 206290 206291 206292 206293 206294 206295 206296 206297 206298 206299 206300 206301 206302 206303 206304 206305 206306 206307 206308 206309 206310 206311 206312 206313 206314 206315 206316 206317 206318 206319 206320 206321 206322 206323 206324 206325 206326 206327 206328 206329 206330 206331 206332 206333 206334 206335 206336 206337 206338 206339 206340 206341 206342 206343 206344 206345 206346 206347 206348 206349 206350 206351 206352 206353 206354 206355 206356 206357 206358 206359 206360 206361 206362 206363 206364 206365 206366 206367 206368 206369 206370 206371 206372 206373 206374 206375 206376 206377 206378 206379 206380 206381 206382 206383 206384 206385 206386 206387 206388 206389 206390 206391 206392 206393 206394 206395 206396 206397 206398 206399 206400 206401 206402 206403 206404 206405 206406 206407 206408 206409 206410 206411 206412 206413 206414 206415 206416 206417 206418 206419 206420 206421 206422 206423 206424 206425 206426 206427 206428 206429 206430 206431 206432 206433 206434 206435 206436 206437 206438 206439 206440 206441 206442 206443 206444 206445 206446 206447 206448 206449 206450 206451 206452 206453 206454 206455 206456 206457 206458 206459 206460 206461 206462 206463 206464 206465 206466 206467 206468 206469 206470 206471 206472 206473 206474 206475 206476 206477 206478 206479 206480 206481 206482 206483 206484 206485 206486 206487 206488 206489 206490 206491 206492 206493 206494 206495 206496 206497 206498 206499 206500 206501 206502 206503 206504 206505 206506 206507 206508 206509 206510 206511 206512 206513 206514 206515 206516 206517 206518 206519 206520 206521 206522 206523 206524 206525 206526 206527 206528 206529 206530 206531 206532 206533 206534 206535 206536 206537 206538 206539 206540 206541 206542 206543 206544 206545 206546 206547 206548 206549 206550 206551 206552 206553 206554 206555 206556 206557 206558 206559 206560 206561 206562 206563 206564 206565 206566 206567 206568 206569 206570 206571 206572 206573 206574 206575 206576 206577 206578 206579 206580 206581 206582 206583 206584 206585 206586 206587 206588 206589 206590 206591 206592 206593 206594 206595 206596 206597 206598 206599 206600 206601 206602 206603 206604 206605 206606 206607 206608 206609 206610 206611 206612 206613 206614 206615 206616 206617 206618 206619 206620 206621 206622 206623 206624 206625 206626 206627 206628 206629 206630 206631 206632 206633 206634 206635 206636 206637 206638 206639 206640 206641 206642 206643 206644 206645 206646 206647 206648 206649 206650 206651 206652 206653 206654 206655 206656 206657 206658 206659 206660 206661 206662 206663 206664 206665 206666 206667 206668 206669 206670 206671 206672 206673 206674 206675 206676 206677 206678 206679 206680 206681 206682 206683 206684 206685 206686 206687 206688 206689 206690 206691 206692 206693 206694 206695 206696 206697 206698 206699 206700 206701 206702 206703 206704 206705 206706 206707 206708 206709 206710 206711 206712 206713 206714 206715 206716 206717 206718 206719 206720 206721 206722 206723 206724 206725 206726 206727 206728 206729 206730 206731 206732 206733 206734 206735 206736 206737 206738 206739 206740 206741 206742 206743 206744 206745 206746 206747 206748 206749 206750 206751 206752 206753 206754 206755 206756 206757 206758 206759 206760 206761 206762 206763 206764 206765 206766 206767 206768 206769 206770 206771 206772 206773 206774 206775 206776 206777 206778 206779 206780 206781 206782 206783 206784 206785 206786 206787 206788 206789 206790 206791 206792 206793 206794 206795 206796 206797 206798 206799 206800 206801 206802 206803 206804 | goto json_parse_restart; } pParse->iErr = i; return -1; } case 'n': { if( strncmp(z+i,"null",4)==0 && !sqlite3Isalnum(z[i+4]) ){ jsonBlobAppendOneByte(pParse, JSONB_NULL); return i+4; } /* fall-through into the default case that checks for NaN */ } default: { u32 k; int nn; c = z[i]; for(k=0; k<sizeof(aNanInfName)/sizeof(aNanInfName[0]); k++){ if( c!=aNanInfName[k].c1 && c!=aNanInfName[k].c2 ) continue; nn = aNanInfName[k].n; if( sqlite3StrNICmp(&z[i], aNanInfName[k].zMatch, nn)!=0 ){ continue; } if( sqlite3Isalnum(z[i+nn]) ) continue; if( aNanInfName[k].eType==JSONB_FLOAT ){ jsonBlobAppendNode(pParse, JSONB_FLOAT, 5, "9e999"); }else{ jsonBlobAppendOneByte(pParse, JSONB_NULL); } pParse->hasNonstd = 1; return i + nn; } pParse->iErr = i; return -1; /* Syntax error */ } } /* End switch(z[i]) */ } /* ** Parse a complete JSON string. Return 0 on success or non-zero if there ** are any errors. If an error occurs, free all memory held by pParse, ** but not pParse itself. ** ** pParse must be initialized to an empty parse object prior to calling ** this routine. */ static int jsonConvertTextToBlob( JsonParse *pParse, /* Initialize and fill this JsonParse object */ sqlite3_context *pCtx /* Report errors here */ ){ int i; const char *zJson = pParse->zJson; i = jsonTranslateTextToBlob(pParse, 0); if( pParse->oom ) i = -1; if( i>0 ){ #ifdef SQLITE_DEBUG assert( pParse->iDepth==0 ); if( sqlite3Config.bJsonSelfcheck ){ assert( jsonbValidityCheck(pParse, 0, pParse->nBlob, 0)==0 ); } #endif while( jsonIsspace(zJson[i]) ) i++; if( zJson[i] ){ i += json5Whitespace(&zJson[i]); if( zJson[i] ){ if( pCtx ) sqlite3_result_error(pCtx, "malformed JSON", -1); jsonParseReset(pParse); return 1; } pParse->hasNonstd = 1; } } if( i<=0 ){ if( pCtx!=0 ){ if( pParse->oom ){ sqlite3_result_error_nomem(pCtx); }else{ sqlite3_result_error(pCtx, "malformed JSON", -1); } } jsonParseReset(pParse); return 1; } return 0; } /* ** The input string pStr is a well-formed JSON text string. Convert ** this into the JSONB format and make it the return value of the ** SQL function. */ static void jsonReturnStringAsBlob(JsonString *pStr){ JsonParse px; memset(&px, 0, sizeof(px)); jsonStringTerminate(pStr); px.zJson = pStr->zBuf; px.nJson = pStr->nUsed; px.db = sqlite3_context_db_handle(pStr->pCtx); (void)jsonTranslateTextToBlob(&px, 0); if( px.oom ){ sqlite3DbFree(px.db, px.aBlob); sqlite3_result_error_nomem(pStr->pCtx); }else{ assert( px.nBlobAlloc>0 ); assert( !px.bReadOnly ); sqlite3_result_blob(pStr->pCtx, px.aBlob, px.nBlob, SQLITE_DYNAMIC); } } /* The byte at index i is a node type-code. This routine ** determines the payload size for that node and writes that ** payload size in to *pSz. It returns the offset from i to the ** beginning of the payload. Return 0 on error. */ static u32 jsonbPayloadSize(const JsonParse *pParse, u32 i, u32 *pSz){ u8 x; u32 sz; u32 n; if( NEVER(i>pParse->nBlob) ){ *pSz = 0; return 0; } x = pParse->aBlob[i]>>4; if( x<=11 ){ sz = x; n = 1; }else if( x==12 ){ if( i+1>=pParse->nBlob ){ *pSz = 0; return 0; } sz = pParse->aBlob[i+1]; n = 2; }else if( x==13 ){ if( i+2>=pParse->nBlob ){ *pSz = 0; return 0; } sz = (pParse->aBlob[i+1]<<8) + pParse->aBlob[i+2]; n = 3; }else if( x==14 ){ if( i+4>=pParse->nBlob ){ *pSz = 0; return 0; } sz = ((u32)pParse->aBlob[i+1]<<24) + (pParse->aBlob[i+2]<<16) + (pParse->aBlob[i+3]<<8) + pParse->aBlob[i+4]; n = 5; }else{ if( i+8>=pParse->nBlob || pParse->aBlob[i+1]!=0 || pParse->aBlob[i+2]!=0 || pParse->aBlob[i+3]!=0 || pParse->aBlob[i+4]!=0 ){ *pSz = 0; return 0; } sz = (pParse->aBlob[i+5]<<24) + (pParse->aBlob[i+6]<<16) + (pParse->aBlob[i+7]<<8) + pParse->aBlob[i+8]; n = 9; } if( i+sz+n > pParse->nBlob && i+sz+n > pParse->nBlob-pParse->delta ){ sz = 0; n = 0; } *pSz = sz; return n; } /* ** Translate the binary JSONB representation of JSON beginning at ** pParse->aBlob[i] into a JSON text string. Append the JSON ** text onto the end of pOut. Return the index in pParse->aBlob[] ** of the first byte past the end of the element that is translated. ** ** If an error is detected in the BLOB input, the pOut->eErr flag ** might get set to JSTRING_MALFORMED. But not all BLOB input errors ** are detected. So a malformed JSONB input might either result ** in an error, or in incorrect JSON. ** ** The pOut->eErr JSTRING_OOM flag is set on a OOM. */ static u32 jsonTranslateBlobToText( const JsonParse *pParse, /* the complete parse of the JSON */ u32 i, /* Start rendering at this index */ JsonString *pOut /* Write JSON here */ ){ u32 sz, n, j, iEnd; n = jsonbPayloadSize(pParse, i, &sz); if( n==0 ){ pOut->eErr |= JSTRING_MALFORMED; return pParse->nBlob+1; } switch( pParse->aBlob[i] & 0x0f ){ case JSONB_NULL: { jsonAppendRawNZ(pOut, "null", 4); return i+1; } case JSONB_TRUE: { jsonAppendRawNZ(pOut, "true", 4); return i+1; } case JSONB_FALSE: { jsonAppendRawNZ(pOut, "false", 5); return i+1; } case JSONB_INT: case JSONB_FLOAT: { jsonAppendRaw(pOut, (const char*)&pParse->aBlob[i+n], sz); break; } case JSONB_INT5: { /* Integer literal in hexadecimal notation */ u32 k = 2; sqlite3_uint64 u = 0; const char *zIn = (const char*)&pParse->aBlob[i+n]; int bOverflow = 0; if( zIn[0]=='-' ){ jsonAppendChar(pOut, '-'); k++; }else if( zIn[0]=='+' ){ k++; } for(; k<sz; k++){ if( !sqlite3Isxdigit(zIn[k]) ){ pOut->eErr |= JSTRING_MALFORMED; break; }else if( (u>>60)!=0 ){ bOverflow = 1; }else{ u = u*16 + sqlite3HexToInt(zIn[k]); } } jsonPrintf(100,pOut,bOverflow?"9.0e999":"%llu", u); break; } case JSONB_FLOAT5: { /* Float literal missing digits beside "." */ u32 k = 0; const char *zIn = (const char*)&pParse->aBlob[i+n]; if( zIn[0]=='-' ){ jsonAppendChar(pOut, '-'); k++; } if( zIn[k]=='.' ){ jsonAppendChar(pOut, '0'); } for(; k<sz; k++){ jsonAppendChar(pOut, zIn[k]); if( zIn[k]=='.' && (k+1==sz || !sqlite3Isdigit(zIn[k+1])) ){ jsonAppendChar(pOut, '0'); } } break; } case JSONB_TEXT: case JSONB_TEXTJ: { jsonAppendChar(pOut, '"'); jsonAppendRaw(pOut, (const char*)&pParse->aBlob[i+n], sz); jsonAppendChar(pOut, '"'); break; } case JSONB_TEXT5: { const char *zIn; u32 k; u32 sz2 = sz; zIn = (const char*)&pParse->aBlob[i+n]; jsonAppendChar(pOut, '"'); while( sz2>0 ){ for(k=0; k<sz2 && zIn[k]!='\\' && zIn[k]!='"'; k++){} if( k>0 ){ jsonAppendRawNZ(pOut, zIn, k); if( k>=sz2 ){ break; } zIn += k; sz2 -= k; } if( zIn[0]=='"' ){ jsonAppendRawNZ(pOut, "\\\"", 2); zIn++; sz2--; continue; } assert( zIn[0]=='\\' ); assert( sz2>=1 ); if( sz2<2 ){ pOut->eErr |= JSTRING_MALFORMED; break; } switch( (u8)zIn[1] ){ case '\'': jsonAppendChar(pOut, '\''); break; case 'v': jsonAppendRawNZ(pOut, "\\u0009", 6); break; case 'x': if( sz2<4 ){ pOut->eErr |= JSTRING_MALFORMED; sz2 = 2; break; } jsonAppendRawNZ(pOut, "\\u00", 4); jsonAppendRawNZ(pOut, &zIn[2], 2); zIn += 2; sz2 -= 2; break; case '0': jsonAppendRawNZ(pOut, "\\u0000", 6); break; case '\r': if( sz2>2 && zIn[2]=='\n' ){ zIn++; sz2--; } break; case '\n': break; case 0xe2: /* '\' followed by either U+2028 or U+2029 is ignored as ** whitespace. Not that in UTF8, U+2028 is 0xe2 0x80 0x29. ** U+2029 is the same except for the last byte */ if( sz2<4 || 0x80!=(u8)zIn[2] || (0xa8!=(u8)zIn[3] && 0xa9!=(u8)zIn[3]) ){ pOut->eErr |= JSTRING_MALFORMED; sz2 = 2; break; } zIn += 2; sz2 -= 2; break; default: jsonAppendRawNZ(pOut, zIn, 2); break; } assert( sz2>=2 ); zIn += 2; sz2 -= 2; } jsonAppendChar(pOut, '"'); break; } case JSONB_TEXTRAW: { jsonAppendString(pOut, (const char*)&pParse->aBlob[i+n], sz); break; } case JSONB_ARRAY: { jsonAppendChar(pOut, '['); j = i+n; iEnd = j+sz; while( j<iEnd ){ j = jsonTranslateBlobToText(pParse, j, pOut); jsonAppendChar(pOut, ','); } if( sz>0 ) pOut->nUsed--; jsonAppendChar(pOut, ']'); break; } case JSONB_OBJECT: { int x = 0; jsonAppendChar(pOut, '{'); j = i+n; iEnd = j+sz; while( j<iEnd ){ j = jsonTranslateBlobToText(pParse, j, pOut); jsonAppendChar(pOut, (x++ & 1) ? ',' : ':'); } if( x & 1 ) pOut->eErr |= JSTRING_MALFORMED; if( sz>0 ) pOut->nUsed--; jsonAppendChar(pOut, '}'); break; } default: { pOut->eErr |= JSTRING_MALFORMED; break; } } return i+n+sz; } /* Return true if the input pJson ** ** For performance reasons, this routine does not do a detailed check of the ** input BLOB to ensure that it is well-formed. Hence, false positives are ** possible. False negatives should never occur, however. */ static int jsonFuncArgMightBeBinary(sqlite3_value *pJson){ u32 sz, n; const u8 *aBlob; int nBlob; JsonParse s; if( sqlite3_value_type(pJson)!=SQLITE_BLOB ) return 0; aBlob = sqlite3_value_blob(pJson); nBlob = sqlite3_value_bytes(pJson); if( nBlob<1 ) return 0; if( NEVER(aBlob==0) || (aBlob[0] & 0x0f)>JSONB_OBJECT ) return 0; memset(&s, 0, sizeof(s)); s.aBlob = (u8*)aBlob; s.nBlob = nBlob; n = jsonbPayloadSize(&s, 0, &sz); if( n==0 ) return 0; if( sz+n!=(u32)nBlob ) return 0; if( (aBlob[0] & 0x0f)<=JSONB_FALSE && sz>0 ) return 0; return sz+n==(u32)nBlob; } /* ** Given that a JSONB_ARRAY object starts at offset i, return ** the number of entries in that array. */ static u32 jsonbArrayCount(JsonParse *pParse, u32 iRoot){ u32 n, sz, i, iEnd; u32 k = 0; n = jsonbPayloadSize(pParse, iRoot, &sz); iEnd = iRoot+n+sz; for(i=iRoot+n; n>0 && i<iEnd; i+=sz+n, k++){ n = jsonbPayloadSize(pParse, i, &sz); } return k; } /* ** Edit the payload size of the element at iRoot by the amount in ** pParse->delta. */ static void jsonAfterEditSizeAdjust(JsonParse *pParse, u32 iRoot){ u32 sz = 0; u32 nBlob; assert( pParse->delta!=0 ); assert( pParse->nBlobAlloc >= pParse->nBlob ); nBlob = pParse->nBlob; pParse->nBlob = pParse->nBlobAlloc; (void)jsonbPayloadSize(pParse, iRoot, &sz); pParse->nBlob = nBlob; sz += pParse->delta; pParse->delta += jsonBlobChangePayloadSize(pParse, iRoot, sz); } /* ** Modify the JSONB blob at pParse->aBlob by removing nDel bytes of ** content beginning at iDel, and replacing them with nIns bytes of ** content given by aIns. ** ** nDel may be zero, in which case no bytes are removed. But iDel is ** still important as new bytes will be insert beginning at iDel. ** ** aIns may be zero, in which case space is created to hold nIns bytes ** beginning at iDel, but that space is uninitialized. ** ** Set pParse->oom if an OOM occurs. */ static void jsonBlobEdit( JsonParse *pParse, /* The JSONB to be modified is in pParse->aBlob */ u32 iDel, /* First byte to be removed */ u32 nDel, /* Number of bytes to remove */ const u8 *aIns, /* Content to insert */ u32 nIns /* Bytes of content to insert */ ){ i64 d = (i64)nIns - (i64)nDel; if( d!=0 ){ if( pParse->nBlob + d > pParse->nBlobAlloc ){ jsonBlobExpand(pParse, pParse->nBlob+d); if( pParse->oom ) return; } memmove(&pParse->aBlob[iDel+nIns], &pParse->aBlob[iDel+nDel], pParse->nBlob - (iDel+nDel)); pParse->nBlob += d; pParse->delta += d; } if( nIns && aIns ) memcpy(&pParse->aBlob[iDel], aIns, nIns); } /* ** Return the number of escaped newlines to be ignored. ** An escaped newline is a one of the following byte sequences: ** ** 0x5c 0x0a ** 0x5c 0x0d ** 0x5c 0x0d 0x0a ** 0x5c 0xe2 0x80 0xa8 ** 0x5c 0xe2 0x80 0xa9 */ static u32 jsonBytesToBypass(const char *z, u32 n){ u32 i = 0; while( i+1<n ){ if( z[i]!='\\' ) return i; if( z[i+1]=='\n' ){ i += 2; continue; } if( z[i+1]=='\r' ){ if( i+2<n && z[i+2]=='\n' ){ i += 3; }else{ i += 2; } continue; } if( 0xe2==(u8)z[i+1] && i+3<n && 0x80==(u8)z[i+2] && (0xa8==(u8)z[i+3] || 0xa9==(u8)z[i+3]) ){ i += 4; continue; } break; } return i; } /* ** Input z[0..n] defines JSON escape sequence including the leading '\\'. ** Decode that escape sequence into a single character. Write that ** character into *piOut. Return the number of bytes in the escape sequence. ** ** If there is a syntax error of some kind (for example too few characters ** after the '\\' to complete the encoding) then *piOut is set to ** JSON_INVALID_CHAR. */ static u32 jsonUnescapeOneChar(const char *z, u32 n, u32 *piOut){ assert( n>0 ); assert( z[0]=='\\' ); if( n<2 ){ *piOut = JSON_INVALID_CHAR; return n; } switch( (u8)z[1] ){ case 'u': { u32 v, vlo; if( n<6 ){ *piOut = JSON_INVALID_CHAR; return n; } v = jsonHexToInt4(&z[2]); if( (v & 0xfc00)==0xd800 && n>=12 && z[6]=='\\' && z[7]=='u' && ((vlo = jsonHexToInt4(&z[8]))&0xfc00)==0xdc00 ){ *piOut = ((v&0x3ff)<<10) + (vlo&0x3ff) + 0x10000; return 12; }else{ *piOut = v; return 6; } } case 'b': { *piOut = '\b'; return 2; } case 'f': { *piOut = '\f'; return 2; } case 'n': { *piOut = '\n'; return 2; } case 'r': { *piOut = '\r'; return 2; } case 't': { *piOut = '\t'; return 2; } case 'v': { *piOut = '\v'; return 2; } case '0': { *piOut = 0; return 2; } case '\'': case '"': case '/': case '\\':{ *piOut = z[1]; return 2; } case 'x': { if( n<4 ){ *piOut = JSON_INVALID_CHAR; return n; } *piOut = (jsonHexToInt(z[2])<<4) | jsonHexToInt(z[3]); return 4; } case 0xe2: case '\r': case '\n': { u32 nSkip = jsonBytesToBypass(z, n); if( nSkip==0 ){ *piOut = JSON_INVALID_CHAR; return n; }else if( nSkip==n ){ *piOut = 0; return n; }else if( z[nSkip]=='\\' ){ return nSkip + jsonUnescapeOneChar(&z[nSkip], n-nSkip, piOut); }else{ int sz = sqlite3Utf8ReadLimited((u8*)&z[nSkip], n-nSkip, piOut); return nSkip + sz; } } default: { *piOut = JSON_INVALID_CHAR; return 2; } } } /* ** Compare two object labels. Return 1 if they are equal and ** 0 if they differ. ** ** In this version, we know that one or the other or both of the ** two comparands contains an escape sequence. */ static SQLITE_NOINLINE int jsonLabelCompareEscaped( const char *zLeft, /* The left label */ u32 nLeft, /* Size of the left label in bytes */ int rawLeft, /* True if zLeft contains no escapes */ const char *zRight, /* The right label */ u32 nRight, /* Size of the right label in bytes */ int rawRight /* True if zRight is escape-free */ ){ u32 cLeft, cRight; assert( rawLeft==0 || rawRight==0 ); while( 1 /*exit-by-return*/ ){ if( nLeft==0 ){ cLeft = 0; }else if( rawLeft || zLeft[0]!='\\' ){ cLeft = ((u8*)zLeft)[0]; if( cLeft>=0xc0 ){ int sz = sqlite3Utf8ReadLimited((u8*)zLeft, nLeft, &cLeft); zLeft += sz; nLeft -= sz; }else{ zLeft++; nLeft--; } }else{ u32 n = jsonUnescapeOneChar(zLeft, nLeft, &cLeft); zLeft += n; assert( n<=nLeft ); nLeft -= n; } if( nRight==0 ){ cRight = 0; }else if( rawRight || zRight[0]!='\\' ){ cRight = ((u8*)zRight)[0]; if( cRight>=0xc0 ){ int sz = sqlite3Utf8ReadLimited((u8*)zRight, nRight, &cRight); zRight += sz; nRight -= sz; }else{ zRight++; nRight--; } }else{ u32 n = jsonUnescapeOneChar(zRight, nRight, &cRight); zRight += n; assert( n<=nRight ); nRight -= n; } if( cLeft!=cRight ) return 0; if( cLeft==0 ) return 1; } } /* ** Compare two object labels. Return 1 if they are equal and ** 0 if they differ. Return -1 if an OOM occurs. */ static int jsonLabelCompare( const char *zLeft, /* The left label */ u32 nLeft, /* Size of the left label in bytes */ int rawLeft, /* True if zLeft contains no escapes */ const char *zRight, /* The right label */ u32 nRight, /* Size of the right label in bytes */ int rawRight /* True if zRight is escape-free */ ){ if( rawLeft && rawRight ){ /* Simpliest case: Neither label contains escapes. A simple ** memcmp() is sufficient. */ if( nLeft!=nRight ) return 0; return memcmp(zLeft, zRight, nLeft)==0; }else{ return jsonLabelCompareEscaped(zLeft, nLeft, rawLeft, zRight, nRight, rawRight); } } /* ** Error returns from jsonLookupStep() */ #define JSON_LOOKUP_ERROR 0xffffffff #define JSON_LOOKUP_NOTFOUND 0xfffffffe #define JSON_LOOKUP_PATHERROR 0xfffffffd #define JSON_LOOKUP_ISERROR(x) ((x)>=JSON_LOOKUP_PATHERROR) /* Forward declaration */ static u32 jsonLookupStep(JsonParse*,u32,const char*,u32); /* This helper routine for jsonLookupStep() populates pIns with ** binary data that is to be inserted into pParse. ** ** In the common case, pIns just points to pParse->aIns and pParse->nIns. ** But if the zPath of the original edit operation includes path elements ** that go deeper, additional substructure must be created. ** ** For example: ** ** json_insert('{}', '$.a.b.c', 123); ** ** The search stops at '$.a' But additional substructure must be ** created for the ".b.c" part of the patch so that the final result ** is: {"a":{"b":{"c"::123}}}. This routine populates pIns with ** the binary equivalent of {"b":{"c":123}} so that it can be inserted. ** ** The caller is responsible for resetting pIns when it has finished ** using the substructure. */ static u32 jsonCreateEditSubstructure( JsonParse *pParse, /* The original JSONB that is being edited */ JsonParse *pIns, /* Populate this with the blob data to insert */ const char *zTail /* Tail of the path that determins substructure */ ){ static const u8 emptyObject[] = { JSONB_ARRAY, JSONB_OBJECT }; int rc; memset(pIns, 0, sizeof(*pIns)); pIns->db = pParse->db; if( zTail[0]==0 ){ /* No substructure. Just insert what is given in pParse. */ pIns->aBlob = pParse->aIns; pIns->nBlob = pParse->nIns; rc = 0; }else{ /* Construct the binary substructure */ pIns->nBlob = 1; pIns->aBlob = (u8*)&emptyObject[zTail[0]=='.']; pIns->eEdit = pParse->eEdit; pIns->nIns = pParse->nIns; pIns->aIns = pParse->aIns; rc = jsonLookupStep(pIns, 0, zTail, 0); pParse->oom |= pIns->oom; } return rc; /* Error code only */ } /* ** Search along zPath to find the Json element specified. Return an ** index into pParse->aBlob[] for the start of that element's value. ** ** If the value found by this routine is the value half of label/value pair ** within an object, then set pPath->iLabel to the start of the corresponding ** label, before returning. ** ** Return one of the JSON_LOOKUP error codes if problems are seen. ** ** This routine will also modify the blob. If pParse->eEdit is one of ** JEDIT_DEL, JEDIT_REPL, JEDIT_INS, or JEDIT_SET, then changes might be ** made to the selected value. If an edit is performed, then the return ** value does not necessarily point to the select element. If an edit ** is performed, the return value is only useful for detecting error ** conditions. */ static u32 jsonLookupStep( JsonParse *pParse, /* The JSON to search */ u32 iRoot, /* Begin the search at this element of aBlob[] */ const char *zPath, /* The path to search */ u32 iLabel /* Label if iRoot is a value of in an object */ ){ u32 i, j, k, nKey, sz, n, iEnd, rc; const char *zKey; u8 x; if( zPath[0]==0 ){ if( pParse->eEdit && jsonBlobMakeEditable(pParse, pParse->nIns) ){ n = jsonbPayloadSize(pParse, iRoot, &sz); sz += n; if( pParse->eEdit==JEDIT_DEL ){ if( iLabel>0 ){ sz += iRoot - iLabel; iRoot = iLabel; } jsonBlobEdit(pParse, iRoot, sz, 0, 0); }else if( pParse->eEdit==JEDIT_INS ){ /* Already exists, so json_insert() is a no-op */ }else{ /* json_set() or json_replace() */ jsonBlobEdit(pParse, iRoot, sz, pParse->aIns, pParse->nIns); } } pParse->iLabel = iLabel; return iRoot; } if( zPath[0]=='.' ){ int rawKey = 1; x = pParse->aBlob[iRoot]; zPath++; if( zPath[0]=='"' ){ zKey = zPath + 1; for(i=1; zPath[i] && zPath[i]!='"'; i++){} nKey = i-1; if( zPath[i] ){ i++; }else{ return JSON_LOOKUP_PATHERROR; } testcase( nKey==0 ); rawKey = memchr(zKey, '\\', nKey)==0; }else{ zKey = zPath; for(i=0; zPath[i] && zPath[i]!='.' && zPath[i]!='['; i++){} nKey = i; if( nKey==0 ){ return JSON_LOOKUP_PATHERROR; } } if( (x & 0x0f)!=JSONB_OBJECT ) return JSON_LOOKUP_NOTFOUND; n = jsonbPayloadSize(pParse, iRoot, &sz); j = iRoot + n; /* j is the index of a label */ iEnd = j+sz; while( j<iEnd ){ int rawLabel; const char *zLabel; x = pParse->aBlob[j] & 0x0f; if( x<JSONB_TEXT || x>JSONB_TEXTRAW ) return JSON_LOOKUP_ERROR; n = jsonbPayloadSize(pParse, j, &sz); if( n==0 ) return JSON_LOOKUP_ERROR; k = j+n; /* k is the index of the label text */ if( k+sz>=iEnd ) return JSON_LOOKUP_ERROR; zLabel = (const char*)&pParse->aBlob[k]; rawLabel = x==JSONB_TEXT || x==JSONB_TEXTRAW; if( jsonLabelCompare(zKey, nKey, rawKey, zLabel, sz, rawLabel) ){ u32 v = k+sz; /* v is the index of the value */ if( ((pParse->aBlob[v])&0x0f)>JSONB_OBJECT ) return JSON_LOOKUP_ERROR; n = jsonbPayloadSize(pParse, v, &sz); if( n==0 || v+n+sz>iEnd ) return JSON_LOOKUP_ERROR; assert( j>0 ); rc = jsonLookupStep(pParse, v, &zPath[i], j); if( pParse->delta ) jsonAfterEditSizeAdjust(pParse, iRoot); return rc; } j = k+sz; if( ((pParse->aBlob[j])&0x0f)>JSONB_OBJECT ) return JSON_LOOKUP_ERROR; n = jsonbPayloadSize(pParse, j, &sz); if( n==0 ) return JSON_LOOKUP_ERROR; j += n+sz; } if( j>iEnd ) return JSON_LOOKUP_ERROR; if( pParse->eEdit>=JEDIT_INS ){ u32 nIns; /* Total bytes to insert (label+value) */ JsonParse v; /* BLOB encoding of the value to be inserted */ JsonParse ix; /* Header of the label to be inserted */ testcase( pParse->eEdit==JEDIT_INS ); testcase( pParse->eEdit==JEDIT_SET ); memset(&ix, 0, sizeof(ix)); ix.db = pParse->db; jsonBlobAppendNode(&ix, rawKey?JSONB_TEXTRAW:JSONB_TEXT5, nKey, 0); pParse->oom |= ix.oom; rc = jsonCreateEditSubstructure(pParse, &v, &zPath[i]); if( !JSON_LOOKUP_ISERROR(rc) && jsonBlobMakeEditable(pParse, ix.nBlob+nKey+v.nBlob) ){ assert( !pParse->oom ); nIns = ix.nBlob + nKey + v.nBlob; jsonBlobEdit(pParse, j, 0, 0, nIns); if( !pParse->oom ){ assert( pParse->aBlob!=0 ); /* Because pParse->oom!=0 */ assert( ix.aBlob!=0 ); /* Because pPasre->oom!=0 */ memcpy(&pParse->aBlob[j], ix.aBlob, ix.nBlob); k = j + ix.nBlob; memcpy(&pParse->aBlob[k], zKey, nKey); k += nKey; memcpy(&pParse->aBlob[k], v.aBlob, v.nBlob); if( ALWAYS(pParse->delta) ) jsonAfterEditSizeAdjust(pParse, iRoot); } } jsonParseReset(&v); jsonParseReset(&ix); return rc; } }else if( zPath[0]=='[' ){ x = pParse->aBlob[iRoot] & 0x0f; if( x!=JSONB_ARRAY ) return JSON_LOOKUP_NOTFOUND; n = jsonbPayloadSize(pParse, iRoot, &sz); k = 0; i = 1; while( sqlite3Isdigit(zPath[i]) ){ k = k*10 + zPath[i] - '0'; i++; } if( i<2 || zPath[i]!=']' ){ if( zPath[1]=='#' ){ k = jsonbArrayCount(pParse, iRoot); i = 2; if( zPath[2]=='-' && sqlite3Isdigit(zPath[3]) ){ unsigned int nn = 0; i = 3; do{ nn = nn*10 + zPath[i] - '0'; i++; }while( sqlite3Isdigit(zPath[i]) ); if( nn>k ) return JSON_LOOKUP_NOTFOUND; k -= nn; } if( zPath[i]!=']' ){ return JSON_LOOKUP_PATHERROR; } }else{ return JSON_LOOKUP_PATHERROR; } } j = iRoot+n; iEnd = j+sz; while( j<iEnd ){ if( k==0 ){ rc = jsonLookupStep(pParse, j, &zPath[i+1], 0); if( pParse->delta ) jsonAfterEditSizeAdjust(pParse, iRoot); return rc; } k--; n = jsonbPayloadSize(pParse, j, &sz); if( n==0 ) return JSON_LOOKUP_ERROR; j += n+sz; } if( j>iEnd ) return JSON_LOOKUP_ERROR; if( k>0 ) return JSON_LOOKUP_NOTFOUND; if( pParse->eEdit>=JEDIT_INS ){ JsonParse v; testcase( pParse->eEdit==JEDIT_INS ); testcase( pParse->eEdit==JEDIT_SET ); rc = jsonCreateEditSubstructure(pParse, &v, &zPath[i+1]); if( !JSON_LOOKUP_ISERROR(rc) && jsonBlobMakeEditable(pParse, v.nBlob) ){ assert( !pParse->oom ); jsonBlobEdit(pParse, j, 0, v.aBlob, v.nBlob); } jsonParseReset(&v); if( pParse->delta ) jsonAfterEditSizeAdjust(pParse, iRoot); return rc; } }else{ return JSON_LOOKUP_PATHERROR; } return JSON_LOOKUP_NOTFOUND; } /* ** Convert a JSON BLOB into text and make that text the return value ** of an SQL function. */ static void jsonReturnTextJsonFromBlob( sqlite3_context *ctx, const u8 *aBlob, u32 nBlob ){ JsonParse x; JsonString s; if( NEVER(aBlob==0) ) return; memset(&x, 0, sizeof(x)); x.aBlob = (u8*)aBlob; x.nBlob = nBlob; jsonStringInit(&s, ctx); jsonTranslateBlobToText(&x, 0, &s); jsonReturnString(&s, 0, 0); } /* ** Return the value of the BLOB node at index i. ** ** If the value is a primitive, return it as an SQL value. ** If the value is an array or object, return it as either ** JSON text or the BLOB encoding, depending on the JSON_B flag ** on the userdata. */ static void jsonReturnFromBlob( JsonParse *pParse, /* Complete JSON parse tree */ u32 i, /* Index of the node */ sqlite3_context *pCtx, /* Return value for this function */ int textOnly /* return text JSON. Disregard user-data */ ){ u32 n, sz; int rc; sqlite3 *db = sqlite3_context_db_handle(pCtx); n = jsonbPayloadSize(pParse, i, &sz); if( n==0 ){ sqlite3_result_error(pCtx, "malformed JSON", -1); return; } switch( pParse->aBlob[i] & 0x0f ){ case JSONB_NULL: { if( sz ) goto returnfromblob_malformed; sqlite3_result_null(pCtx); break; } case JSONB_TRUE: { if( sz ) goto returnfromblob_malformed; sqlite3_result_int(pCtx, 1); break; } case JSONB_FALSE: { if( sz ) goto returnfromblob_malformed; sqlite3_result_int(pCtx, 0); break; } case JSONB_INT5: case JSONB_INT: { sqlite3_int64 iRes = 0; char *z; int bNeg = 0; char x; if( sz==0 ) goto returnfromblob_malformed; x = (char)pParse->aBlob[i+n]; if( x=='-' ){ if( sz<2 ) goto returnfromblob_malformed; n++; sz--; bNeg = 1; } z = sqlite3DbStrNDup(db, (const char*)&pParse->aBlob[i+n], (int)sz); if( z==0 ) goto returnfromblob_oom; rc = sqlite3DecOrHexToI64(z, &iRes); sqlite3DbFree(db, z); if( rc==0 ){ sqlite3_result_int64(pCtx, bNeg ? -iRes : iRes); }else if( rc==3 && bNeg ){ sqlite3_result_int64(pCtx, SMALLEST_INT64); }else if( rc==1 ){ goto returnfromblob_malformed; }else{ if( bNeg ){ n--; sz++; } goto to_double; } break; } case JSONB_FLOAT5: case JSONB_FLOAT: { double r; char *z; if( sz==0 ) goto returnfromblob_malformed; to_double: z = sqlite3DbStrNDup(db, (const char*)&pParse->aBlob[i+n], (int)sz); if( z==0 ) goto returnfromblob_oom; rc = sqlite3AtoF(z, &r, sqlite3Strlen30(z), SQLITE_UTF8); sqlite3DbFree(db, z); if( rc<=0 ) goto returnfromblob_malformed; sqlite3_result_double(pCtx, r); break; } case JSONB_TEXTRAW: case JSONB_TEXT: { sqlite3_result_text(pCtx, (char*)&pParse->aBlob[i+n], sz, SQLITE_TRANSIENT); break; } case JSONB_TEXT5: case JSONB_TEXTJ: { /* Translate JSON formatted string into raw text */ u32 iIn, iOut; const char *z; char *zOut; u32 nOut = sz; z = (const char*)&pParse->aBlob[i+n]; zOut = sqlite3DbMallocRaw(db, nOut+1); if( zOut==0 ) goto returnfromblob_oom; for(iIn=iOut=0; iIn<sz; iIn++){ char c = z[iIn]; if( c=='\\' ){ u32 v; u32 szEscape = jsonUnescapeOneChar(&z[iIn], sz-iIn, &v); if( v<=0x7f ){ zOut[iOut++] = (char)v; }else if( v<=0x7ff ){ assert( szEscape>=2 ); zOut[iOut++] = (char)(0xc0 | (v>>6)); zOut[iOut++] = 0x80 | (v&0x3f); }else if( v<0x10000 ){ assert( szEscape>=3 ); zOut[iOut++] = 0xe0 | (v>>12); zOut[iOut++] = 0x80 | ((v>>6)&0x3f); zOut[iOut++] = 0x80 | (v&0x3f); }else if( v==JSON_INVALID_CHAR ){ /* Silently ignore illegal unicode */ }else{ assert( szEscape>=4 ); zOut[iOut++] = 0xf0 | (v>>18); zOut[iOut++] = 0x80 | ((v>>12)&0x3f); zOut[iOut++] = 0x80 | ((v>>6)&0x3f); zOut[iOut++] = 0x80 | (v&0x3f); } iIn += szEscape - 1; }else{ zOut[iOut++] = c; } } /* end for() */ assert( iOut<=nOut ); zOut[iOut] = 0; sqlite3_result_text(pCtx, zOut, iOut, SQLITE_DYNAMIC); break; } case JSONB_ARRAY: case JSONB_OBJECT: { int flags = textOnly ? 0 : SQLITE_PTR_TO_INT(sqlite3_user_data(pCtx)); if( flags & JSON_BLOB ){ sqlite3_result_blob(pCtx, &pParse->aBlob[i], sz+n, SQLITE_TRANSIENT); }else{ jsonReturnTextJsonFromBlob(pCtx, &pParse->aBlob[i], sz+n); } break; } default: { goto returnfromblob_malformed; } } return; returnfromblob_oom: sqlite3_result_error_nomem(pCtx); return; returnfromblob_malformed: sqlite3_result_error(pCtx, "malformed JSON", -1); return; } /* ** pArg is a function argument that might be an SQL value or a JSON ** value. Figure out what it is and encode it as a JSONB blob. ** Return the results in pParse. ** ** pParse is uninitialized upon entry. This routine will handle the ** initialization of pParse. The result will be contained in ** pParse->aBlob and pParse->nBlob. pParse->aBlob might be dynamically ** allocated (if pParse->nBlobAlloc is greater than zero) in which case ** the caller is responsible for freeing the space allocated to pParse->aBlob ** when it has finished with it. Or pParse->aBlob might be a static string ** or a value obtained from sqlite3_value_blob(pArg). ** ** If the argument is a BLOB that is clearly not a JSONB, then this ** function might set an error message in ctx and return non-zero. ** It might also set an error message and return non-zero on an OOM error. */ static int jsonFunctionArgToBlob( sqlite3_context *ctx, sqlite3_value *pArg, JsonParse *pParse ){ int eType = sqlite3_value_type(pArg); static u8 aNull[] = { 0x00 }; memset(pParse, 0, sizeof(pParse[0])); pParse->db = sqlite3_context_db_handle(ctx); switch( eType ){ default: { pParse->aBlob = aNull; pParse->nBlob = 1; return 0; } case SQLITE_BLOB: { if( jsonFuncArgMightBeBinary(pArg) ){ pParse->aBlob = (u8*)sqlite3_value_blob(pArg); pParse->nBlob = sqlite3_value_bytes(pArg); }else{ sqlite3_result_error(ctx, "JSON cannot hold BLOB values", -1); return 1; } break; } case SQLITE_TEXT: { const char *zJson = (const char*)sqlite3_value_text(pArg); int nJson = sqlite3_value_bytes(pArg); if( zJson==0 ) return 1; if( sqlite3_value_subtype(pArg)==JSON_SUBTYPE ){ pParse->zJson = (char*)zJson; pParse->nJson = nJson; if( jsonConvertTextToBlob(pParse, ctx) ){ sqlite3_result_error(ctx, "malformed JSON", -1); sqlite3DbFree(pParse->db, pParse->aBlob); memset(pParse, 0, sizeof(pParse[0])); return 1; } }else{ jsonBlobAppendNode(pParse, JSONB_TEXTRAW, nJson, zJson); } break; } case SQLITE_FLOAT: { double r = sqlite3_value_double(pArg); if( NEVER(sqlite3IsNaN(r)) ){ jsonBlobAppendNode(pParse, JSONB_NULL, 0, 0); }else{ int n = sqlite3_value_bytes(pArg); const char *z = (const char*)sqlite3_value_text(pArg); if( z==0 ) return 1; if( z[0]=='I' ){ jsonBlobAppendNode(pParse, JSONB_FLOAT, 5, "9e999"); }else if( z[0]=='-' && z[1]=='I' ){ jsonBlobAppendNode(pParse, JSONB_FLOAT, 6, "-9e999"); }else{ jsonBlobAppendNode(pParse, JSONB_FLOAT, n, z); } } break; } case SQLITE_INTEGER: { int n = sqlite3_value_bytes(pArg); const char *z = (const char*)sqlite3_value_text(pArg); if( z==0 ) return 1; jsonBlobAppendNode(pParse, JSONB_INT, n, z); break; } } if( pParse->oom ){ sqlite3_result_error_nomem(ctx); return 1; }else{ return 0; } } /* ** Generate a bad path error. ** ** If ctx is not NULL then push the error message into ctx and return NULL. ** If ctx is NULL, then return the text of the error message. */ static char *jsonBadPathError( sqlite3_context *ctx, /* The function call containing the error */ const char *zPath /* The path with the problem */ ){ char *zMsg = sqlite3_mprintf("bad JSON path: %Q", zPath); if( ctx==0 ) return zMsg; if( zMsg ){ sqlite3_result_error(ctx, zMsg, -1); sqlite3_free(zMsg); }else{ sqlite3_result_error_nomem(ctx); } return 0; } /* argv[0] is a BLOB that seems likely to be a JSONB. Subsequent ** arguments come in parse where each pair contains a JSON path and ** content to insert or set at that patch. Do the updates ** and return the result. ** ** The specific operation is determined by eEdit, which can be one ** of JEDIT_INS, JEDIT_REPL, or JEDIT_SET. */ static void jsonInsertIntoBlob( sqlite3_context *ctx, int argc, sqlite3_value **argv, int eEdit /* JEDIT_INS, JEDIT_REPL, or JEDIT_SET */ ){ int i; u32 rc = 0; const char *zPath = 0; int flgs; JsonParse *p; JsonParse ax; assert( (argc&1)==1 ); flgs = argc==1 ? 0 : JSON_EDITABLE; p = jsonParseFuncArg(ctx, argv[0], flgs); if( p==0 ) return; for(i=1; i<argc-1; i+=2){ if( sqlite3_value_type(argv[i])==SQLITE_NULL ) continue; zPath = (const char*)sqlite3_value_text(argv[i]); if( zPath==0 ){ sqlite3_result_error_nomem(ctx); jsonParseFree(p); return; } if( zPath[0]!='$' ) goto jsonInsertIntoBlob_patherror; if( jsonFunctionArgToBlob(ctx, argv[i+1], &ax) ){ jsonParseReset(&ax); jsonParseFree(p); return; } if( zPath[1]==0 ){ if( eEdit==JEDIT_REPL || eEdit==JEDIT_SET ){ jsonBlobEdit(p, 0, p->nBlob, ax.aBlob, ax.nBlob); } rc = 0; }else{ p->eEdit = eEdit; p->nIns = ax.nBlob; p->aIns = ax.aBlob; p->delta = 0; rc = jsonLookupStep(p, 0, zPath+1, 0); } jsonParseReset(&ax); if( rc==JSON_LOOKUP_NOTFOUND ) continue; if( JSON_LOOKUP_ISERROR(rc) ) goto jsonInsertIntoBlob_patherror; } jsonReturnParse(ctx, p); jsonParseFree(p); return; jsonInsertIntoBlob_patherror: jsonParseFree(p); if( rc==JSON_LOOKUP_ERROR ){ sqlite3_result_error(ctx, "malformed JSON", -1); }else{ jsonBadPathError(ctx, zPath); } return; } /* ** Generate a JsonParse object, containing valid JSONB in aBlob and nBlob, ** from the SQL function argument pArg. Return a pointer to the new ** JsonParse object. ** ** Ownership of the new JsonParse object is passed to the caller. The ** caller should invoke jsonParseFree() on the return value when it ** has finished using it. ** ** If any errors are detected, an appropriate error messages is set ** using sqlite3_result_error() or the equivalent and this routine ** returns NULL. This routine also returns NULL if the pArg argument ** is an SQL NULL value, but no error message is set in that case. This ** is so that SQL functions that are given NULL arguments will return ** a NULL value. */ static JsonParse *jsonParseFuncArg( sqlite3_context *ctx, sqlite3_value *pArg, u32 flgs ){ int eType; /* Datatype of pArg */ JsonParse *p = 0; /* Value to be returned */ JsonParse *pFromCache = 0; /* Value taken from cache */ sqlite3 *db; /* The database connection */ assert( ctx!=0 ); eType = sqlite3_value_type(pArg); if( eType==SQLITE_NULL ){ return 0; } pFromCache = jsonCacheSearch(ctx, pArg); if( pFromCache ){ pFromCache->nJPRef++; if( (flgs & JSON_EDITABLE)==0 ){ return pFromCache; } } db = sqlite3_context_db_handle(ctx); rebuild_from_cache: p = sqlite3DbMallocZero(db, sizeof(*p)); if( p==0 ) goto json_pfa_oom; memset(p, 0, sizeof(*p)); p->db = db; p->nJPRef = 1; if( pFromCache!=0 ){ u32 nBlob = pFromCache->nBlob; p->aBlob = sqlite3DbMallocRaw(db, nBlob); if( p->aBlob==0 ) goto json_pfa_oom; memcpy(p->aBlob, pFromCache->aBlob, nBlob); p->nBlobAlloc = p->nBlob = nBlob; p->hasNonstd = pFromCache->hasNonstd; jsonParseFree(pFromCache); return p; } if( eType==SQLITE_BLOB ){ u32 n, sz = 0; p->aBlob = (u8*)sqlite3_value_blob(pArg); p->nBlob = (u32)sqlite3_value_bytes(pArg); if( p->nBlob==0 ){ goto json_pfa_malformed; } if( NEVER(p->aBlob==0) ){ goto json_pfa_oom; } if( (p->aBlob[0] & 0x0f)>JSONB_OBJECT ){ goto json_pfa_malformed; } n = jsonbPayloadSize(p, 0, &sz); if( n==0 || sz+n!=p->nBlob || ((p->aBlob[0] & 0x0f)<=JSONB_FALSE && sz>0) ){ goto json_pfa_malformed; } if( (flgs & JSON_EDITABLE)!=0 && jsonBlobMakeEditable(p, 0)==0 ){ goto json_pfa_oom; } return p; } p->zJson = (char*)sqlite3_value_text(pArg); p->nJson = sqlite3_value_bytes(pArg); if( p->nJson==0 ) goto json_pfa_malformed; if( NEVER(p->zJson==0) ) goto json_pfa_oom; if( jsonConvertTextToBlob(p, (flgs & JSON_KEEPERROR) ? 0 : ctx) ){ if( flgs & JSON_KEEPERROR ){ p->nErr = 1; return p; }else{ jsonParseFree(p); return 0; } }else{ int isRCStr = sqlite3ValueIsOfClass(pArg, sqlite3RCStrUnref); int rc; if( !isRCStr ){ char *zNew = sqlite3RCStrNew( p->nJson ); if( zNew==0 ) goto json_pfa_oom; memcpy(zNew, p->zJson, p->nJson); p->zJson = zNew; p->zJson[p->nJson] = 0; }else{ sqlite3RCStrRef(p->zJson); } p->bJsonIsRCStr = 1; rc = jsonCacheInsert(ctx, p); if( rc==SQLITE_NOMEM ) goto json_pfa_oom; if( flgs & JSON_EDITABLE ){ pFromCache = p; p = 0; goto rebuild_from_cache; } } return p; json_pfa_malformed: if( flgs & JSON_KEEPERROR ){ p->nErr = 1; return p; }else{ jsonParseFree(p); sqlite3_result_error(ctx, "malformed JSON", -1); return 0; } json_pfa_oom: jsonParseFree(pFromCache); jsonParseFree(p); sqlite3_result_error_nomem(ctx); return 0; } /* ** Make the return value of a JSON function either the raw JSONB blob ** or make it JSON text, depending on whether the JSON_BLOB flag is ** set on the function. */ static void jsonReturnParse( sqlite3_context *ctx, JsonParse *p ){ int flgs; if( p->oom ){ sqlite3_result_error_nomem(ctx); return; } flgs = SQLITE_PTR_TO_INT(sqlite3_user_data(ctx)); if( flgs & JSON_BLOB ){ if( p->nBlobAlloc>0 && !p->bReadOnly ){ sqlite3_result_blob(ctx, p->aBlob, p->nBlob, SQLITE_DYNAMIC); p->nBlobAlloc = 0; }else{ sqlite3_result_blob(ctx, p->aBlob, p->nBlob, SQLITE_TRANSIENT); } }else{ JsonString s; jsonStringInit(&s, ctx); p->delta = 0; jsonTranslateBlobToText(p, 0, &s); jsonReturnString(&s, p, ctx); sqlite3_result_subtype(ctx, JSON_SUBTYPE); } } /**************************************************************************** ** SQL functions used for testing and debugging ****************************************************************************/ #if SQLITE_DEBUG /* ** Decode JSONB bytes in aBlob[] starting at iStart through but not ** including iEnd. Indent the ** content by nIndent spaces. */ static void jsonDebugPrintBlob( JsonParse *pParse, /* JSON content */ u32 iStart, /* Start rendering here */ u32 iEnd, /* Do not render this byte or any byte after this one */ int nIndent, /* Indent by this many spaces */ sqlite3_str *pOut /* Generate output into this sqlite3_str object */ ){ while( iStart<iEnd ){ u32 i, n, nn, sz = 0; int showContent = 1; u8 x = pParse->aBlob[iStart] & 0x0f; u32 savedNBlob = pParse->nBlob; sqlite3_str_appendf(pOut, "%5d:%*s", iStart, nIndent, ""); if( pParse->nBlobAlloc>pParse->nBlob ){ pParse->nBlob = pParse->nBlobAlloc; } nn = n = jsonbPayloadSize(pParse, iStart, &sz); if( nn==0 ) nn = 1; if( sz>0 && x<JSONB_ARRAY ){ nn += sz; } for(i=0; i<nn; i++){ sqlite3_str_appendf(pOut, " %02x", pParse->aBlob[iStart+i]); } if( n==0 ){ sqlite3_str_appendf(pOut, " ERROR invalid node size\n"); iStart = n==0 ? iStart+1 : iEnd; continue; } pParse->nBlob = savedNBlob; if( iStart+n+sz>iEnd ){ iEnd = iStart+n+sz; if( iEnd>pParse->nBlob ){ if( pParse->nBlobAlloc>0 && iEnd>pParse->nBlobAlloc ){ iEnd = pParse->nBlobAlloc; }else{ iEnd = pParse->nBlob; } } } sqlite3_str_appendall(pOut," <-- "); switch( x ){ case JSONB_NULL: sqlite3_str_appendall(pOut,"null"); break; case JSONB_TRUE: sqlite3_str_appendall(pOut,"true"); break; case JSONB_FALSE: sqlite3_str_appendall(pOut,"false"); break; case JSONB_INT: sqlite3_str_appendall(pOut,"int"); break; case JSONB_INT5: sqlite3_str_appendall(pOut,"int5"); break; case JSONB_FLOAT: sqlite3_str_appendall(pOut,"float"); break; case JSONB_FLOAT5: sqlite3_str_appendall(pOut,"float5"); break; case JSONB_TEXT: sqlite3_str_appendall(pOut,"text"); break; case JSONB_TEXTJ: sqlite3_str_appendall(pOut,"textj"); break; case JSONB_TEXT5: sqlite3_str_appendall(pOut,"text5"); break; case JSONB_TEXTRAW: sqlite3_str_appendall(pOut,"textraw"); break; case JSONB_ARRAY: { sqlite3_str_appendf(pOut,"array, %u bytes\n", sz); jsonDebugPrintBlob(pParse, iStart+n, iStart+n+sz, nIndent+2, pOut); showContent = 0; break; } case JSONB_OBJECT: { sqlite3_str_appendf(pOut, "object, %u bytes\n", sz); jsonDebugPrintBlob(pParse, iStart+n, iStart+n+sz, nIndent+2, pOut); showContent = 0; break; } default: { sqlite3_str_appendall(pOut, "ERROR: unknown node type\n"); showContent = 0; break; } } if( showContent ){ if( sz==0 && x<=JSONB_FALSE ){ sqlite3_str_append(pOut, "\n", 1); }else{ u32 i; sqlite3_str_appendall(pOut, ": \""); for(i=iStart+n; i<iStart+n+sz; i++){ u8 c = pParse->aBlob[i]; if( c<0x20 || c>=0x7f ) c = '.'; sqlite3_str_append(pOut, (char*)&c, 1); } sqlite3_str_append(pOut, "\"\n", 2); } } iStart += n + sz; } } static void jsonShowParse(JsonParse *pParse){ sqlite3_str out; char zBuf[1000]; if( pParse==0 ){ printf("NULL pointer\n"); return; }else{ printf("nBlobAlloc = %u\n", pParse->nBlobAlloc); printf("nBlob = %u\n", pParse->nBlob); printf("delta = %d\n", pParse->delta); if( pParse->nBlob==0 ) return; printf("content (bytes 0..%u):\n", pParse->nBlob-1); } sqlite3StrAccumInit(&out, 0, zBuf, sizeof(zBuf), 1000000); jsonDebugPrintBlob(pParse, 0, pParse->nBlob, 0, &out); printf("%s", sqlite3_str_value(&out)); sqlite3_str_reset(&out); } #endif /* SQLITE_DEBUG */ #ifdef SQLITE_DEBUG /* ** SQL function: json_parse(JSON) ** ** Parse JSON using jsonParseFuncArg(). Return text that is a ** human-readable dump of the binary JSONB for the input parameter. */ static void jsonParseFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p; /* The parse */ sqlite3_str out; assert( argc>=1 ); sqlite3StrAccumInit(&out, 0, 0, 0, 1000000); p = jsonParseFuncArg(ctx, argv[0], 0); if( p==0 ) return; if( argc==1 ){ jsonDebugPrintBlob(p, 0, p->nBlob, 0, &out); sqlite3_result_text64(ctx, out.zText, out.nChar, SQLITE_DYNAMIC, SQLITE_UTF8); }else{ jsonShowParse(p); } jsonParseFree(p); } #endif /* SQLITE_DEBUG */ /**************************************************************************** ** Scalar SQL function implementations ****************************************************************************/ /* ** Implementation of the json_quote(VALUE) function. Return a JSON value ** corresponding to the SQL value input. Mostly this means putting ** double-quotes around strings and returning the unquoted string "null" ** when given a NULL input. */ static void jsonQuoteFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonString jx; UNUSED_PARAMETER(argc); jsonStringInit(&jx, ctx); jsonAppendSqlValue(&jx, argv[0]); jsonReturnString(&jx, 0, 0); sqlite3_result_subtype(ctx, JSON_SUBTYPE); } /* ** Implementation of the json_array(VALUE,...) function. Return a JSON ** array that contains all values given in arguments. Or if any argument ** is a BLOB, throw an error. */ static void jsonArrayFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ int i; JsonString jx; jsonStringInit(&jx, ctx); jsonAppendChar(&jx, '['); for(i=0; i<argc; i++){ jsonAppendSeparator(&jx); jsonAppendSqlValue(&jx, argv[i]); } jsonAppendChar(&jx, ']'); jsonReturnString(&jx, 0, 0); sqlite3_result_subtype(ctx, JSON_SUBTYPE); } /* ** json_array_length(JSON) ** json_array_length(JSON, PATH) ** ** Return the number of elements in the top-level JSON array. ** Return 0 if the input is not a well-formed JSON array. */ static void jsonArrayLengthFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p; /* The parse */ sqlite3_int64 cnt = 0; u32 i; u8 eErr = 0; p = jsonParseFuncArg(ctx, argv[0], 0); if( p==0 ) return; if( argc==2 ){ const char *zPath = (const char*)sqlite3_value_text(argv[1]); if( zPath==0 ){ jsonParseFree(p); return; } i = jsonLookupStep(p, 0, zPath[0]=='$' ? zPath+1 : "@", 0); if( JSON_LOOKUP_ISERROR(i) ){ if( i==JSON_LOOKUP_NOTFOUND ){ /* no-op */ }else if( i==JSON_LOOKUP_PATHERROR ){ jsonBadPathError(ctx, zPath); }else{ sqlite3_result_error(ctx, "malformed JSON", -1); } eErr = 1; i = 0; } }else{ i = 0; } if( (p->aBlob[i] & 0x0f)==JSONB_ARRAY ){ cnt = jsonbArrayCount(p, i); } if( !eErr ) sqlite3_result_int64(ctx, cnt); jsonParseFree(p); } /* True if the string is all digits */ static int jsonAllDigits(const char *z, int n){ int i; for(i=0; i<n && sqlite3Isdigit(z[i]); i++){} return i==n; } /* True if the string is all alphanumerics and underscores */ static int jsonAllAlphanum(const char *z, int n){ int i; for(i=0; i<n && (sqlite3Isalnum(z[i]) || z[i]=='_'); i++){} return i==n; } /* ** json_extract(JSON, PATH, ...) ** "->"(JSON,PATH) ** "->>"(JSON,PATH) ** ** Return the element described by PATH. Return NULL if that PATH element |
︙ | ︙ | |||
205054 205055 205056 205057 205058 205059 205060 | ** compatibility with PG. */ static void jsonExtractFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ | | < < | > | | > > | > > > | > > | > > > | < | | | | | | | | | | | | | > > > > > | | | | | > | | | | > | > | > > | > > > | > > > | | | | | < > | > > > | < < < < < < < < < | > > | > | | > | | | | | | > > > | > > | | > > > > > > > > | | > > > > > | < > > > > > > > | > | < | > > > | > | | | < > > > > > > > > > > > | > > | > > > | > | | | > > > > > > > > > > > > > > | < < > > > | > > > > > | | | | > > > > | | | | | | > | | | < | < | | < | > > > > > > | > | > > > > | | > > > > > > > > | < | < > | > > > > | > > | > > > > > > > > > > > > > > > | > > | > > > > > > > > | < > > > > > > > > | > > > | < < > > > > > | | > > | | | > > | > | | | > | | < < | | < < | < | | > | < | | | > > > | 206817 206818 206819 206820 206821 206822 206823 206824 206825 206826 206827 206828 206829 206830 206831 206832 206833 206834 206835 206836 206837 206838 206839 206840 206841 206842 206843 206844 206845 206846 206847 206848 206849 206850 206851 206852 206853 206854 206855 206856 206857 206858 206859 206860 206861 206862 206863 206864 206865 206866 206867 206868 206869 206870 206871 206872 206873 206874 206875 206876 206877 206878 206879 206880 206881 206882 206883 206884 206885 206886 206887 206888 206889 206890 206891 206892 206893 206894 206895 206896 206897 206898 206899 206900 206901 206902 206903 206904 206905 206906 206907 206908 206909 206910 206911 206912 206913 206914 206915 206916 206917 206918 206919 206920 206921 206922 206923 206924 206925 206926 206927 206928 206929 206930 206931 206932 206933 206934 206935 206936 206937 206938 206939 206940 206941 206942 206943 206944 206945 206946 206947 206948 206949 206950 206951 206952 206953 206954 206955 206956 206957 206958 206959 206960 206961 206962 206963 206964 206965 206966 206967 206968 206969 206970 206971 206972 206973 206974 206975 206976 206977 206978 206979 206980 206981 206982 206983 206984 206985 206986 206987 206988 206989 206990 206991 206992 206993 206994 206995 206996 206997 206998 206999 207000 207001 207002 207003 207004 207005 207006 207007 207008 207009 207010 207011 207012 207013 207014 207015 207016 207017 207018 207019 207020 207021 207022 207023 207024 207025 207026 207027 207028 207029 207030 207031 207032 207033 207034 207035 207036 207037 207038 207039 207040 207041 207042 207043 207044 207045 207046 207047 207048 207049 207050 207051 207052 207053 207054 207055 207056 207057 207058 207059 207060 207061 207062 207063 207064 207065 207066 207067 207068 207069 207070 207071 207072 207073 207074 207075 207076 207077 207078 207079 207080 207081 207082 207083 207084 207085 207086 207087 207088 207089 207090 207091 207092 207093 207094 207095 207096 207097 207098 207099 207100 207101 207102 207103 207104 207105 207106 207107 207108 207109 207110 207111 207112 207113 207114 207115 207116 207117 207118 207119 207120 207121 207122 207123 207124 207125 207126 207127 207128 207129 207130 207131 207132 207133 207134 207135 207136 207137 207138 207139 207140 207141 207142 207143 207144 207145 207146 207147 207148 207149 207150 207151 207152 207153 207154 207155 207156 207157 207158 207159 207160 207161 207162 | ** compatibility with PG. */ static void jsonExtractFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p = 0; /* The parse */ int flags; /* Flags associated with the function */ int i; /* Loop counter */ JsonString jx; /* String for array result */ if( argc<2 ) return; p = jsonParseFuncArg(ctx, argv[0], 0); if( p==0 ) return; flags = SQLITE_PTR_TO_INT(sqlite3_user_data(ctx)); jsonStringInit(&jx, ctx); if( argc>2 ){ jsonAppendChar(&jx, '['); } for(i=1; i<argc; i++){ /* With a single PATH argument */ const char *zPath = (const char*)sqlite3_value_text(argv[i]); int nPath; u32 j; if( zPath==0 ) goto json_extract_error; nPath = sqlite3Strlen30(zPath); if( zPath[0]=='$' ){ j = jsonLookupStep(p, 0, zPath+1, 0); }else if( (flags & JSON_ABPATH) ){ /* The -> and ->> operators accept abbreviated PATH arguments. This ** is mostly for compatibility with PostgreSQL, but also for ** convenience. ** ** NUMBER ==> $[NUMBER] // PG compatible ** LABEL ==> $.LABEL // PG compatible ** [NUMBER] ==> $[NUMBER] // Not PG. Purely for convenience */ jsonStringInit(&jx, ctx); if( jsonAllDigits(zPath, nPath) ){ jsonAppendRawNZ(&jx, "[", 1); jsonAppendRaw(&jx, zPath, nPath); jsonAppendRawNZ(&jx, "]", 2); }else if( jsonAllAlphanum(zPath, nPath) ){ jsonAppendRawNZ(&jx, ".", 1); jsonAppendRaw(&jx, zPath, nPath); }else if( zPath[0]=='[' && nPath>=3 && zPath[nPath-1]==']' ){ jsonAppendRaw(&jx, zPath, nPath); }else{ jsonAppendRawNZ(&jx, ".\"", 2); jsonAppendRaw(&jx, zPath, nPath); jsonAppendRawNZ(&jx, "\"", 1); } jsonStringTerminate(&jx); j = jsonLookupStep(p, 0, jx.zBuf, 0); jsonStringReset(&jx); }else{ jsonBadPathError(ctx, zPath); goto json_extract_error; } if( j<p->nBlob ){ if( argc==2 ){ if( flags & JSON_JSON ){ jsonStringInit(&jx, ctx); jsonTranslateBlobToText(p, j, &jx); jsonReturnString(&jx, 0, 0); jsonStringReset(&jx); assert( (flags & JSON_BLOB)==0 ); sqlite3_result_subtype(ctx, JSON_SUBTYPE); }else{ jsonReturnFromBlob(p, j, ctx, 0); if( (flags & (JSON_SQL|JSON_BLOB))==0 && (p->aBlob[j]&0x0f)>=JSONB_ARRAY ){ sqlite3_result_subtype(ctx, JSON_SUBTYPE); } } }else{ jsonAppendSeparator(&jx); jsonTranslateBlobToText(p, j, &jx); } }else if( j==JSON_LOOKUP_NOTFOUND ){ if( argc==2 ){ goto json_extract_error; /* Return NULL if not found */ }else{ jsonAppendSeparator(&jx); jsonAppendRawNZ(&jx, "null", 4); } }else if( j==JSON_LOOKUP_ERROR ){ sqlite3_result_error(ctx, "malformed JSON", -1); goto json_extract_error; }else{ jsonBadPathError(ctx, zPath); goto json_extract_error; } } if( argc>2 ){ jsonAppendChar(&jx, ']'); jsonReturnString(&jx, 0, 0); if( (flags & JSON_BLOB)==0 ){ sqlite3_result_subtype(ctx, JSON_SUBTYPE); } } json_extract_error: jsonStringReset(&jx); jsonParseFree(p); return; } /* ** Return codes for jsonMergePatch() */ #define JSON_MERGE_OK 0 /* Success */ #define JSON_MERGE_BADTARGET 1 /* Malformed TARGET blob */ #define JSON_MERGE_BADPATCH 2 /* Malformed PATCH blob */ #define JSON_MERGE_OOM 3 /* Out-of-memory condition */ /* ** RFC-7396 MergePatch for two JSONB blobs. ** ** pTarget is the target. pPatch is the patch. The target is updated ** in place. The patch is read-only. ** ** The original RFC-7396 algorithm is this: ** ** define MergePatch(Target, Patch): ** if Patch is an Object: ** if Target is not an Object: ** Target = {} # Ignore the contents and set it to an empty Object ** for each Name/Value pair in Patch: ** if Value is null: ** if Name exists in Target: ** remove the Name/Value pair from Target ** else: ** Target[Name] = MergePatch(Target[Name], Value) ** return Target ** else: ** return Patch ** ** Here is an equivalent algorithm restructured to show the actual ** implementation: ** ** 01 define MergePatch(Target, Patch): ** 02 if Patch is not an Object: ** 03 return Patch ** 04 else: // if Patch is an Object ** 05 if Target is not an Object: ** 06 Target = {} ** 07 for each Name/Value pair in Patch: ** 08 if Name exists in Target: ** 09 if Value is null: ** 10 remove the Name/Value pair from Target ** 11 else ** 12 Target[name] = MergePatch(Target[Name], Value) ** 13 else if Value is not NULL: ** 14 if Value is not an Object: ** 15 Target[name] = Value ** 16 else: ** 17 Target[name] = MergePatch('{}',value) ** 18 return Target ** | ** ^---- Line numbers referenced in comments in the implementation */ static int jsonMergePatch( JsonParse *pTarget, /* The JSON parser that contains the TARGET */ u32 iTarget, /* Index of TARGET in pTarget->aBlob[] */ const JsonParse *pPatch, /* The PATCH */ u32 iPatch /* Index of PATCH in pPatch->aBlob[] */ ){ u8 x; /* Type of a single node */ u32 n, sz=0; /* Return values from jsonbPayloadSize() */ u32 iTCursor; /* Cursor position while scanning the target object */ u32 iTStart; /* First label in the target object */ u32 iTEndBE; /* Original first byte past end of target, before edit */ u32 iTEnd; /* Current first byte past end of target */ u8 eTLabel; /* Node type of the target label */ u32 iTLabel = 0; /* Index of the label */ u32 nTLabel = 0; /* Header size in bytes for the target label */ u32 szTLabel = 0; /* Size of the target label payload */ u32 iTValue = 0; /* Index of the target value */ u32 nTValue = 0; /* Header size of the target value */ u32 szTValue = 0; /* Payload size for the target value */ u32 iPCursor; /* Cursor position while scanning the patch */ u32 iPEnd; /* First byte past the end of the patch */ u8 ePLabel; /* Node type of the patch label */ u32 iPLabel; /* Start of patch label */ u32 nPLabel; /* Size of header on the patch label */ u32 szPLabel; /* Payload size of the patch label */ u32 iPValue; /* Start of patch value */ u32 nPValue; /* Header size for the patch value */ u32 szPValue; /* Payload size of the patch value */ assert( iTarget>=0 && iTarget<pTarget->nBlob ); assert( iPatch>=0 && iPatch<pPatch->nBlob ); x = pPatch->aBlob[iPatch] & 0x0f; if( x!=JSONB_OBJECT ){ /* Algorithm line 02 */ u32 szPatch; /* Total size of the patch, header+payload */ u32 szTarget; /* Total size of the target, header+payload */ n = jsonbPayloadSize(pPatch, iPatch, &sz); szPatch = n+sz; sz = 0; n = jsonbPayloadSize(pTarget, iTarget, &sz); szTarget = n+sz; jsonBlobEdit(pTarget, iTarget, szTarget, pPatch->aBlob+iPatch, szPatch); return pTarget->oom ? JSON_MERGE_OOM : JSON_MERGE_OK; /* Line 03 */ } x = pTarget->aBlob[iTarget] & 0x0f; if( x!=JSONB_OBJECT ){ /* Algorithm line 05 */ n = jsonbPayloadSize(pTarget, iTarget, &sz); jsonBlobEdit(pTarget, iTarget+n, sz, 0, 0); x = pTarget->aBlob[iTarget]; pTarget->aBlob[iTarget] = (x & 0xf0) | JSONB_OBJECT; } n = jsonbPayloadSize(pPatch, iPatch, &sz); if( NEVER(n==0) ) return JSON_MERGE_BADPATCH; iPCursor = iPatch+n; iPEnd = iPCursor+sz; n = jsonbPayloadSize(pTarget, iTarget, &sz); if( NEVER(n==0) ) return JSON_MERGE_BADTARGET; iTStart = iTarget+n; iTEndBE = iTStart+sz; while( iPCursor<iPEnd ){ /* Algorithm line 07 */ iPLabel = iPCursor; ePLabel = pPatch->aBlob[iPCursor] & 0x0f; if( ePLabel<JSONB_TEXT || ePLabel>JSONB_TEXTRAW ){ return JSON_MERGE_BADPATCH; } nPLabel = jsonbPayloadSize(pPatch, iPCursor, &szPLabel); if( nPLabel==0 ) return JSON_MERGE_BADPATCH; iPValue = iPCursor + nPLabel + szPLabel; if( iPValue>=iPEnd ) return JSON_MERGE_BADPATCH; nPValue = jsonbPayloadSize(pPatch, iPValue, &szPValue); if( nPValue==0 ) return JSON_MERGE_BADPATCH; iPCursor = iPValue + nPValue + szPValue; if( iPCursor>iPEnd ) return JSON_MERGE_BADPATCH; iTCursor = iTStart; iTEnd = iTEndBE + pTarget->delta; while( iTCursor<iTEnd ){ int isEqual; /* true if the patch and target labels match */ iTLabel = iTCursor; eTLabel = pTarget->aBlob[iTCursor] & 0x0f; if( eTLabel<JSONB_TEXT || eTLabel>JSONB_TEXTRAW ){ return JSON_MERGE_BADTARGET; } nTLabel = jsonbPayloadSize(pTarget, iTCursor, &szTLabel); if( nTLabel==0 ) return JSON_MERGE_BADTARGET; iTValue = iTLabel + nTLabel + szTLabel; if( iTValue>=iTEnd ) return JSON_MERGE_BADTARGET; nTValue = jsonbPayloadSize(pTarget, iTValue, &szTValue); if( nTValue==0 ) return JSON_MERGE_BADTARGET; if( iTValue + nTValue + szTValue > iTEnd ) return JSON_MERGE_BADTARGET; isEqual = jsonLabelCompare( (const char*)&pPatch->aBlob[iPLabel+nPLabel], szPLabel, (ePLabel==JSONB_TEXT || ePLabel==JSONB_TEXTRAW), (const char*)&pTarget->aBlob[iTLabel+nTLabel], szTLabel, (eTLabel==JSONB_TEXT || eTLabel==JSONB_TEXTRAW)); if( isEqual ) break; iTCursor = iTValue + nTValue + szTValue; } x = pPatch->aBlob[iPValue] & 0x0f; if( iTCursor<iTEnd ){ /* A match was found. Algorithm line 08 */ if( x==0 ){ /* Patch value is NULL. Algorithm line 09 */ jsonBlobEdit(pTarget, iTLabel, nTLabel+szTLabel+nTValue+szTValue, 0,0); /* vvvvvv----- No OOM on a delete-only edit */ if( NEVER(pTarget->oom) ) return JSON_MERGE_OOM; }else{ /* Algorithm line 12 */ int rc, savedDelta = pTarget->delta; pTarget->delta = 0; rc = jsonMergePatch(pTarget, iTValue, pPatch, iPValue); if( rc ) return rc; pTarget->delta += savedDelta; } }else if( x>0 ){ /* Algorithm line 13 */ /* No match and patch value is not NULL */ u32 szNew = szPLabel+nPLabel; if( (pPatch->aBlob[iPValue] & 0x0f)!=JSONB_OBJECT ){ /* Line 14 */ jsonBlobEdit(pTarget, iTEnd, 0, 0, szPValue+nPValue+szNew); if( pTarget->oom ) return JSON_MERGE_OOM; memcpy(&pTarget->aBlob[iTEnd], &pPatch->aBlob[iPLabel], szNew); memcpy(&pTarget->aBlob[iTEnd+szNew], &pPatch->aBlob[iPValue], szPValue+nPValue); }else{ int rc, savedDelta; jsonBlobEdit(pTarget, iTEnd, 0, 0, szNew+1); if( pTarget->oom ) return JSON_MERGE_OOM; memcpy(&pTarget->aBlob[iTEnd], &pPatch->aBlob[iPLabel], szNew); pTarget->aBlob[iTEnd+szNew] = 0x00; savedDelta = pTarget->delta; pTarget->delta = 0; rc = jsonMergePatch(pTarget, iTEnd+szNew,pPatch,iPValue); if( rc ) return rc; pTarget->delta += savedDelta; } } } if( pTarget->delta ) jsonAfterEditSizeAdjust(pTarget, iTarget); return pTarget->oom ? JSON_MERGE_OOM : JSON_MERGE_OK; } /* ** Implementation of the json_mergepatch(JSON1,JSON2) function. Return a JSON ** object that is the result of running the RFC 7396 MergePatch() algorithm ** on the two arguments. */ static void jsonPatchFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *pTarget; /* The TARGET */ JsonParse *pPatch; /* The PATCH */ int rc; /* Result code */ UNUSED_PARAMETER(argc); assert( argc==2 ); pTarget = jsonParseFuncArg(ctx, argv[0], JSON_EDITABLE); if( pTarget==0 ) return; pPatch = jsonParseFuncArg(ctx, argv[1], 0); if( pPatch ){ rc = jsonMergePatch(pTarget, 0, pPatch, 0); if( rc==JSON_MERGE_OK ){ jsonReturnParse(ctx, pTarget); }else if( rc==JSON_MERGE_OOM ){ sqlite3_result_error_nomem(ctx); }else{ sqlite3_result_error(ctx, "malformed JSON", -1); } jsonParseFree(pPatch); } jsonParseFree(pTarget); } /* ** Implementation of the json_object(NAME,VALUE,...) function. Return a JSON ** object that contains all name/value given in arguments. Or if any name ** is not a string or if any value is a BLOB, throw an error. |
︙ | ︙ | |||
205255 205256 205257 205258 205259 205260 205261 | u32 n; if( argc&1 ){ sqlite3_result_error(ctx, "json_object() requires an even number " "of arguments", -1); return; } | | | | | | | < | > | | | | | < | < < < < > > | | | < | < | | < < < < < < | < < | | < < < < < | < < < < | | < < < < | < < < < < < < < < | < < < | < | < < < < < | | < < < < < < < < < < | | < < < | < < | | | < < < < | < | | < < | < < | < < < < < < < | < < < < < < < < < < < < < < < | < < < | | | < < < < < < < < < < < < < < < < < < < < | | | > > > > > | > > > > > | < > | < | | > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > | > > > > > > > > | > | | | | > > > > > > > > > > > > > > > > > > > > > > > > | > > | | > > > > > | | | > | | > > | < < < < > | < | < > | < | < < < < < | < < < < > | > > > | < | | | > | > > | > > > | < > | < > | | | > > > > > > | < | > | | | | > > | > > > > | | > > > > | 207172 207173 207174 207175 207176 207177 207178 207179 207180 207181 207182 207183 207184 207185 207186 207187 207188 207189 207190 207191 207192 207193 207194 207195 207196 207197 207198 207199 207200 207201 207202 207203 207204 207205 207206 207207 207208 207209 207210 207211 207212 207213 207214 207215 207216 207217 207218 207219 207220 207221 207222 207223 207224 207225 207226 207227 207228 207229 207230 207231 207232 207233 207234 207235 207236 207237 207238 207239 207240 207241 207242 207243 207244 207245 207246 207247 207248 207249 207250 207251 207252 207253 207254 207255 207256 207257 207258 207259 207260 207261 207262 207263 207264 207265 207266 207267 207268 207269 207270 207271 207272 207273 207274 207275 207276 207277 207278 207279 207280 207281 207282 207283 207284 207285 207286 207287 207288 207289 207290 207291 207292 207293 207294 207295 207296 207297 207298 207299 207300 207301 207302 207303 207304 207305 207306 207307 207308 207309 207310 207311 207312 207313 207314 207315 207316 207317 207318 207319 207320 207321 207322 207323 207324 207325 207326 207327 207328 207329 207330 207331 207332 207333 207334 207335 207336 207337 207338 207339 207340 207341 207342 207343 207344 207345 207346 207347 207348 207349 207350 207351 207352 207353 207354 207355 207356 207357 207358 207359 207360 207361 207362 207363 207364 207365 207366 207367 207368 207369 207370 207371 207372 207373 207374 207375 207376 207377 207378 207379 207380 207381 207382 207383 207384 207385 207386 207387 207388 207389 207390 207391 207392 207393 207394 207395 207396 207397 207398 207399 207400 207401 207402 207403 207404 207405 207406 207407 207408 207409 207410 207411 207412 207413 207414 207415 207416 207417 207418 207419 207420 207421 207422 207423 207424 207425 207426 207427 207428 207429 207430 207431 207432 207433 207434 207435 207436 207437 207438 207439 207440 207441 207442 207443 207444 207445 207446 207447 207448 207449 207450 207451 207452 207453 207454 207455 207456 207457 207458 207459 207460 207461 207462 207463 207464 207465 207466 207467 207468 207469 207470 207471 207472 207473 207474 207475 207476 207477 207478 207479 207480 207481 207482 207483 207484 207485 207486 207487 207488 207489 207490 207491 207492 207493 207494 207495 207496 207497 207498 207499 207500 207501 207502 207503 207504 207505 207506 207507 207508 207509 207510 207511 207512 207513 207514 207515 207516 207517 207518 207519 207520 207521 207522 207523 207524 207525 207526 207527 207528 207529 207530 207531 207532 207533 207534 207535 207536 207537 207538 207539 207540 207541 207542 207543 207544 207545 207546 207547 207548 207549 207550 207551 207552 207553 207554 207555 207556 207557 207558 207559 207560 207561 207562 207563 207564 207565 207566 207567 207568 207569 207570 207571 207572 207573 207574 207575 207576 207577 207578 207579 207580 207581 207582 207583 207584 | u32 n; if( argc&1 ){ sqlite3_result_error(ctx, "json_object() requires an even number " "of arguments", -1); return; } jsonStringInit(&jx, ctx); jsonAppendChar(&jx, '{'); for(i=0; i<argc; i+=2){ if( sqlite3_value_type(argv[i])!=SQLITE_TEXT ){ sqlite3_result_error(ctx, "json_object() labels must be TEXT", -1); jsonStringReset(&jx); return; } jsonAppendSeparator(&jx); z = (const char*)sqlite3_value_text(argv[i]); n = sqlite3_value_bytes(argv[i]); jsonAppendString(&jx, z, n); jsonAppendChar(&jx, ':'); jsonAppendSqlValue(&jx, argv[i+1]); } jsonAppendChar(&jx, '}'); jsonReturnString(&jx, 0, 0); sqlite3_result_subtype(ctx, JSON_SUBTYPE); } /* ** json_remove(JSON, PATH, ...) ** ** Remove the named elements from JSON and return the result. malformed ** JSON or PATH arguments result in an error. */ static void jsonRemoveFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p; /* The parse */ const char *zPath = 0; /* Path of element to be removed */ int i; /* Loop counter */ u32 rc; /* Subroutine return code */ if( argc<1 ) return; p = jsonParseFuncArg(ctx, argv[0], argc>1 ? JSON_EDITABLE : 0); if( p==0 ) return; for(i=1; i<argc; i++){ zPath = (const char*)sqlite3_value_text(argv[i]); if( zPath==0 ){ goto json_remove_done; } if( zPath[0]!='$' ){ goto json_remove_patherror; } if( zPath[1]==0 ){ /* json_remove(j,'$') returns NULL */ goto json_remove_done; } p->eEdit = JEDIT_DEL; p->delta = 0; rc = jsonLookupStep(p, 0, zPath+1, 0); if( JSON_LOOKUP_ISERROR(rc) ){ if( rc==JSON_LOOKUP_NOTFOUND ){ continue; /* No-op */ }else if( rc==JSON_LOOKUP_PATHERROR ){ jsonBadPathError(ctx, zPath); }else{ sqlite3_result_error(ctx, "malformed JSON", -1); } goto json_remove_done; } } jsonReturnParse(ctx, p); jsonParseFree(p); return; json_remove_patherror: jsonBadPathError(ctx, zPath); json_remove_done: jsonParseFree(p); return; } /* ** json_replace(JSON, PATH, VALUE, ...) ** ** Replace the value at PATH with VALUE. If PATH does not already exist, ** this routine is a no-op. If JSON or PATH is malformed, throw an error. */ static void jsonReplaceFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ if( argc<1 ) return; if( (argc&1)==0 ) { jsonWrongNumArgs(ctx, "replace"); return; } jsonInsertIntoBlob(ctx, argc, argv, JEDIT_REPL); } /* ** json_set(JSON, PATH, VALUE, ...) ** ** Set the value at PATH to VALUE. Create the PATH if it does not already ** exist. Overwrite existing values that do exist. ** If JSON or PATH is malformed, throw an error. ** ** json_insert(JSON, PATH, VALUE, ...) ** ** Create PATH and initialize it to VALUE. If PATH already exists, this ** routine is a no-op. If JSON or PATH is malformed, throw an error. */ static void jsonSetFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ int flags = SQLITE_PTR_TO_INT(sqlite3_user_data(ctx)); int bIsSet = (flags&JSON_ISSET)!=0; if( argc<1 ) return; if( (argc&1)==0 ) { jsonWrongNumArgs(ctx, bIsSet ? "set" : "insert"); return; } jsonInsertIntoBlob(ctx, argc, argv, bIsSet ? JEDIT_SET : JEDIT_INS); } /* ** json_type(JSON) ** json_type(JSON, PATH) ** ** Return the top-level "type" of a JSON string. json_type() raises an ** error if either the JSON or PATH inputs are not well-formed. */ static void jsonTypeFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p; /* The parse */ const char *zPath = 0; u32 i; p = jsonParseFuncArg(ctx, argv[0], 0); if( p==0 ) return; if( argc==2 ){ zPath = (const char*)sqlite3_value_text(argv[1]); if( zPath==0 ) goto json_type_done; if( zPath[0]!='$' ){ jsonBadPathError(ctx, zPath); goto json_type_done; } i = jsonLookupStep(p, 0, zPath+1, 0); if( JSON_LOOKUP_ISERROR(i) ){ if( i==JSON_LOOKUP_NOTFOUND ){ /* no-op */ }else if( i==JSON_LOOKUP_PATHERROR ){ jsonBadPathError(ctx, zPath); }else{ sqlite3_result_error(ctx, "malformed JSON", -1); } goto json_type_done; } }else{ i = 0; } sqlite3_result_text(ctx, jsonbType[p->aBlob[i]&0x0f], -1, SQLITE_STATIC); json_type_done: jsonParseFree(p); } /* ** json_valid(JSON) ** json_valid(JSON, FLAGS) ** ** Check the JSON argument to see if it is well-formed. The FLAGS argument ** encodes the various constraints on what is meant by "well-formed": ** ** 0x01 Canonical RFC-8259 JSON text ** 0x02 JSON text with optional JSON-5 extensions ** 0x04 Superficially appears to be JSONB ** 0x08 Strictly well-formed JSONB ** ** If the FLAGS argument is omitted, it defaults to 1. Useful values for ** FLAGS include: ** ** 1 Strict canonical JSON text ** 2 JSON text perhaps with JSON-5 extensions ** 4 Superficially appears to be JSONB ** 5 Canonical JSON text or superficial JSONB ** 6 JSON-5 text or superficial JSONB ** 8 Strict JSONB ** 9 Canonical JSON text or strict JSONB ** 10 JSON-5 text or strict JSONB ** ** Other flag combinations are redundant. For example, every canonical ** JSON text is also well-formed JSON-5 text, so FLAG values 2 and 3 ** are the same. Similarly, any input that passes a strict JSONB validation ** will also pass the superficial validation so 12 through 15 are the same ** as 8 through 11 respectively. ** ** This routine runs in linear time to validate text and when doing strict ** JSONB validation. Superficial JSONB validation is constant time, ** assuming the BLOB is already in memory. The performance advantage ** of superficial JSONB validation is why that option is provided. ** Application developers can choose to do fast superficial validation or ** slower strict validation, according to their specific needs. ** ** Only the lower four bits of the FLAGS argument are currently used. ** Higher bits are reserved for future expansion. To facilitate ** compatibility, the current implementation raises an error if any bit ** in FLAGS is set other than the lower four bits. ** ** The original circa 2015 implementation of the JSON routines in ** SQLite only supported canonical RFC-8259 JSON text and the json_valid() ** function only accepted one argument. That is why the default value ** for the FLAGS argument is 1, since FLAGS=1 causes this routine to only ** recognize canonical RFC-8259 JSON text as valid. The extra FLAGS ** argument was added when the JSON routines were extended to support ** JSON5-like extensions and binary JSONB stored in BLOBs. ** ** Return Values: ** ** * Raise an error if FLAGS is outside the range of 1 to 15. ** * Return NULL if the input is NULL ** * Return 1 if the input is well-formed. ** * Return 0 if the input is not well-formed. */ static void jsonValidFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonParse *p; /* The parse */ u8 flags = 1; u8 res = 0; if( argc==2 ){ i64 f = sqlite3_value_int64(argv[1]); if( f<1 || f>15 ){ sqlite3_result_error(ctx, "FLAGS parameter to json_valid() must be" " between 1 and 15", -1); return; } flags = f & 0x0f; } switch( sqlite3_value_type(argv[0]) ){ case SQLITE_NULL: { #ifdef SQLITE_LEGACY_JSON_VALID /* Incorrect legacy behavior was to return FALSE for a NULL input */ sqlite3_result_int(ctx, 0); #endif return; } case SQLITE_BLOB: { if( (flags & 0x0c)!=0 && jsonFuncArgMightBeBinary(argv[0]) ){ if( flags & 0x04 ){ /* Superficial checking only - accomplished by the ** jsonFuncArgMightBeBinary() call above. */ res = 1; }else{ /* Strict checking. Check by translating BLOB->TEXT->BLOB. If ** no errors occur, call that a "strict check". */ JsonParse px; u32 iErr; memset(&px, 0, sizeof(px)); px.aBlob = (u8*)sqlite3_value_blob(argv[0]); px.nBlob = sqlite3_value_bytes(argv[0]); iErr = jsonbValidityCheck(&px, 0, px.nBlob, 1); res = iErr==0; } } break; } default: { JsonParse px; if( (flags & 0x3)==0 ) break; memset(&px, 0, sizeof(px)); p = jsonParseFuncArg(ctx, argv[0], JSON_KEEPERROR); if( p ){ if( p->oom ){ sqlite3_result_error_nomem(ctx); }else if( p->nErr ){ /* no-op */ }else if( (flags & 0x02)!=0 || p->hasNonstd==0 ){ res = 1; } jsonParseFree(p); }else{ sqlite3_result_error_nomem(ctx); } break; } } sqlite3_result_int(ctx, res); } /* ** json_error_position(JSON) ** ** If the argument is NULL, return NULL ** ** If the argument is BLOB, do a full validity check and return non-zero ** if the check fails. The return value is the approximate 1-based offset ** to the byte of the element that contains the first error. ** ** Otherwise interpret the argument is TEXT (even if it is numeric) and ** return the 1-based character position for where the parser first recognized ** that the input was not valid JSON, or return 0 if the input text looks ** ok. JSON-5 extensions are accepted. */ static void jsonErrorFunc( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ i64 iErrPos = 0; /* Error position to be returned */ JsonParse s; assert( argc==1 ); UNUSED_PARAMETER(argc); memset(&s, 0, sizeof(s)); s.db = sqlite3_context_db_handle(ctx); if( jsonFuncArgMightBeBinary(argv[0]) ){ s.aBlob = (u8*)sqlite3_value_blob(argv[0]); s.nBlob = sqlite3_value_bytes(argv[0]); iErrPos = (i64)jsonbValidityCheck(&s, 0, s.nBlob, 1); }else{ s.zJson = (char*)sqlite3_value_text(argv[0]); if( s.zJson==0 ) return; /* NULL input or OOM */ s.nJson = sqlite3_value_bytes(argv[0]); if( jsonConvertTextToBlob(&s,0) ){ if( s.oom ){ iErrPos = -1; }else{ /* Convert byte-offset s.iErr into a character offset */ u32 k; assert( s.zJson!=0 ); /* Because s.oom is false */ for(k=0; k<s.iErr && ALWAYS(s.zJson[k]); k++){ if( (s.zJson[k] & 0xc0)!=0x80 ) iErrPos++; } iErrPos++; } } } jsonParseReset(&s); if( iErrPos<0 ){ sqlite3_result_error_nomem(ctx); }else{ sqlite3_result_int64(ctx, iErrPos); } } /**************************************************************************** ** Aggregate SQL function implementations ****************************************************************************/ /* ** json_group_array(VALUE) ** ** Return a JSON array composed of all values in the aggregate. */ static void jsonArrayStep( sqlite3_context *ctx, int argc, sqlite3_value **argv ){ JsonString *pStr; UNUSED_PARAMETER(argc); pStr = (JsonString*)sqlite3_aggregate_context(ctx, sizeof(*pStr)); if( pStr ){ if( pStr->zBuf==0 ){ jsonStringInit(pStr, ctx); jsonAppendChar(pStr, '['); }else if( pStr->nUsed>1 ){ jsonAppendChar(pStr, ','); } pStr->pCtx = ctx; jsonAppendSqlValue(pStr, argv[0]); } } static void jsonArrayCompute(sqlite3_context *ctx, int isFinal){ JsonString *pStr; pStr = (JsonString*)sqlite3_aggregate_context(ctx, 0); if( pStr ){ int flags; pStr->pCtx = ctx; jsonAppendChar(pStr, ']'); flags = SQLITE_PTR_TO_INT(sqlite3_user_data(ctx)); if( pStr->eErr ){ jsonReturnString(pStr, 0, 0); return; }else if( flags & JSON_BLOB ){ jsonReturnStringAsBlob(pStr); if( isFinal ){ if( !pStr->bStatic ) sqlite3RCStrUnref(pStr->zBuf); }else{ pStr->nUsed--; } return; }else if( isFinal ){ sqlite3_result_text(ctx, pStr->zBuf, (int)pStr->nUsed, pStr->bStatic ? SQLITE_TRANSIENT : sqlite3RCStrUnref); pStr->bStatic = 1; }else{ sqlite3_result_text(ctx, pStr->zBuf, (int)pStr->nUsed, SQLITE_TRANSIENT); |
︙ | ︙ | |||
205733 205734 205735 205736 205737 205738 205739 | JsonString *pStr; const char *z; u32 n; UNUSED_PARAMETER(argc); pStr = (JsonString*)sqlite3_aggregate_context(ctx, sizeof(*pStr)); if( pStr ){ if( pStr->zBuf==0 ){ | | | | > > > | > > > > | | > > > > | 207659 207660 207661 207662 207663 207664 207665 207666 207667 207668 207669 207670 207671 207672 207673 207674 207675 207676 207677 207678 207679 207680 207681 207682 207683 207684 207685 207686 207687 207688 207689 207690 207691 207692 207693 207694 207695 207696 207697 207698 207699 207700 207701 207702 207703 207704 | JsonString *pStr; const char *z; u32 n; UNUSED_PARAMETER(argc); pStr = (JsonString*)sqlite3_aggregate_context(ctx, sizeof(*pStr)); if( pStr ){ if( pStr->zBuf==0 ){ jsonStringInit(pStr, ctx); jsonAppendChar(pStr, '{'); }else if( pStr->nUsed>1 ){ jsonAppendChar(pStr, ','); } pStr->pCtx = ctx; z = (const char*)sqlite3_value_text(argv[0]); n = sqlite3Strlen30(z); jsonAppendString(pStr, z, n); jsonAppendChar(pStr, ':'); jsonAppendSqlValue(pStr, argv[1]); } } static void jsonObjectCompute(sqlite3_context *ctx, int isFinal){ JsonString *pStr; pStr = (JsonString*)sqlite3_aggregate_context(ctx, 0); if( pStr ){ int flags; jsonAppendChar(pStr, '}'); pStr->pCtx = ctx; flags = SQLITE_PTR_TO_INT(sqlite3_user_data(ctx)); if( pStr->eErr ){ jsonReturnString(pStr, 0, 0); return; }else if( flags & JSON_BLOB ){ jsonReturnStringAsBlob(pStr); if( isFinal ){ if( !pStr->bStatic ) sqlite3RCStrUnref(pStr->zBuf); }else{ pStr->nUsed--; } return; }else if( isFinal ){ sqlite3_result_text(ctx, pStr->zBuf, (int)pStr->nUsed, pStr->bStatic ? SQLITE_TRANSIENT : sqlite3RCStrUnref); pStr->bStatic = 1; }else{ sqlite3_result_text(ctx, pStr->zBuf, (int)pStr->nUsed, SQLITE_TRANSIENT); |
︙ | ︙ | |||
205781 205782 205783 205784 205785 205786 205787 205788 205789 205790 205791 | #ifndef SQLITE_OMIT_VIRTUALTABLE /**************************************************************************** ** The json_each virtual table ****************************************************************************/ typedef struct JsonEachCursor JsonEachCursor; struct JsonEachCursor { sqlite3_vtab_cursor base; /* Base class - must be first */ u32 iRowid; /* The rowid */ | > > > > > > > > > < | > | | > > | > > > > > > > | | 207718 207719 207720 207721 207722 207723 207724 207725 207726 207727 207728 207729 207730 207731 207732 207733 207734 207735 207736 207737 207738 207739 207740 207741 207742 207743 207744 207745 207746 207747 207748 207749 207750 207751 207752 207753 207754 207755 207756 207757 207758 207759 207760 207761 207762 207763 207764 207765 207766 207767 207768 207769 207770 207771 207772 | #ifndef SQLITE_OMIT_VIRTUALTABLE /**************************************************************************** ** The json_each virtual table ****************************************************************************/ typedef struct JsonParent JsonParent; struct JsonParent { u32 iHead; /* Start of object or array */ u32 iValue; /* Start of the value */ u32 iEnd; /* First byte past the end */ u32 nPath; /* Length of path */ i64 iKey; /* Key for JSONB_ARRAY */ }; typedef struct JsonEachCursor JsonEachCursor; struct JsonEachCursor { sqlite3_vtab_cursor base; /* Base class - must be first */ u32 iRowid; /* The rowid */ u32 i; /* Index in sParse.aBlob[] of current row */ u32 iEnd; /* EOF when i equals or exceeds this value */ u32 nRoot; /* Size of the root path in bytes */ u8 eType; /* Type of the container for element i */ u8 bRecursive; /* True for json_tree(). False for json_each() */ u32 nParent; /* Current nesting depth */ u32 nParentAlloc; /* Space allocated for aParent[] */ JsonParent *aParent; /* Parent elements of i */ sqlite3 *db; /* Database connection */ JsonString path; /* Current path */ JsonParse sParse; /* Parse of the input JSON */ }; typedef struct JsonEachConnection JsonEachConnection; struct JsonEachConnection { sqlite3_vtab base; /* Base class - must be first */ sqlite3 *db; /* Database connection */ }; /* Constructor for the json_each virtual table */ static int jsonEachConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ JsonEachConnection *pNew; int rc; /* Column numbers */ #define JEACH_KEY 0 #define JEACH_VALUE 1 #define JEACH_TYPE 2 #define JEACH_ATOM 3 |
︙ | ︙ | |||
205829 205830 205831 205832 205833 205834 205835 | UNUSED_PARAMETER(argv); UNUSED_PARAMETER(argc); UNUSED_PARAMETER(pAux); rc = sqlite3_declare_vtab(db, "CREATE TABLE x(key,value,type,atom,id,parent,fullkey,path," "json HIDDEN,root HIDDEN)"); if( rc==SQLITE_OK ){ | > | < > > | > | | > < > > > > > < < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < < > > | | | < | > | | > | > | | < | | < < | < < < | | < | > | | | > > < > | < | > < > | > > | | < < < < < < < < | | < < | | < | < < < < < < | > > > > > < > > | > > | < < | > > | < | > > | < > > | > > | | | > > | < < < < < < < < < < < < | > > | < | | > | | | < | | | > | > > > < > > > > > > > | | | | > | | | | > | < | | < < < < < | < < | < < | > | < < < < < < < | | > | | < < < < < < | > > > | | > | 207784 207785 207786 207787 207788 207789 207790 207791 207792 207793 207794 207795 207796 207797 207798 207799 207800 207801 207802 207803 207804 207805 207806 207807 207808 207809 207810 207811 207812 207813 207814 207815 207816 207817 207818 207819 207820 207821 207822 207823 207824 207825 207826 207827 207828 207829 207830 207831 207832 207833 207834 207835 207836 207837 207838 207839 207840 207841 207842 207843 207844 207845 207846 207847 207848 207849 207850 207851 207852 207853 207854 207855 207856 207857 207858 207859 207860 207861 207862 207863 207864 207865 207866 207867 207868 207869 207870 207871 207872 207873 207874 207875 207876 207877 207878 207879 207880 207881 207882 207883 207884 207885 207886 207887 207888 207889 207890 207891 207892 207893 207894 207895 207896 207897 207898 207899 207900 207901 207902 207903 207904 207905 207906 207907 207908 207909 207910 207911 207912 207913 207914 207915 207916 207917 207918 207919 207920 207921 207922 207923 207924 207925 207926 207927 207928 207929 207930 207931 207932 207933 207934 207935 207936 207937 207938 207939 207940 207941 207942 207943 207944 207945 207946 207947 207948 207949 207950 207951 207952 207953 207954 207955 207956 207957 207958 207959 207960 207961 207962 207963 207964 207965 207966 207967 207968 207969 207970 207971 207972 207973 207974 207975 207976 207977 207978 207979 207980 207981 207982 207983 207984 207985 207986 207987 207988 207989 207990 207991 207992 207993 207994 207995 207996 207997 207998 207999 208000 208001 208002 208003 208004 208005 208006 208007 208008 208009 208010 208011 208012 208013 208014 208015 208016 208017 208018 208019 208020 208021 208022 208023 208024 208025 208026 208027 208028 208029 208030 208031 208032 208033 208034 208035 208036 208037 208038 208039 208040 208041 208042 208043 208044 208045 208046 208047 208048 208049 208050 208051 208052 208053 208054 208055 208056 208057 208058 208059 208060 208061 208062 208063 208064 208065 208066 208067 208068 208069 208070 208071 208072 208073 208074 208075 208076 208077 208078 208079 208080 208081 208082 208083 208084 208085 208086 208087 208088 208089 208090 208091 208092 | UNUSED_PARAMETER(argv); UNUSED_PARAMETER(argc); UNUSED_PARAMETER(pAux); rc = sqlite3_declare_vtab(db, "CREATE TABLE x(key,value,type,atom,id,parent,fullkey,path," "json HIDDEN,root HIDDEN)"); if( rc==SQLITE_OK ){ pNew = (JsonEachConnection*)sqlite3DbMallocZero(db, sizeof(*pNew)); *ppVtab = (sqlite3_vtab*)pNew; if( pNew==0 ) return SQLITE_NOMEM; sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); pNew->db = db; } return rc; } /* destructor for json_each virtual table */ static int jsonEachDisconnect(sqlite3_vtab *pVtab){ JsonEachConnection *p = (JsonEachConnection*)pVtab; sqlite3DbFree(p->db, pVtab); return SQLITE_OK; } /* constructor for a JsonEachCursor object for json_each(). */ static int jsonEachOpenEach(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ JsonEachConnection *pVtab = (JsonEachConnection*)p; JsonEachCursor *pCur; UNUSED_PARAMETER(p); pCur = sqlite3DbMallocZero(pVtab->db, sizeof(*pCur)); if( pCur==0 ) return SQLITE_NOMEM; pCur->db = pVtab->db; jsonStringZero(&pCur->path); *ppCursor = &pCur->base; return SQLITE_OK; } /* constructor for a JsonEachCursor object for json_tree(). */ static int jsonEachOpenTree(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ int rc = jsonEachOpenEach(p, ppCursor); if( rc==SQLITE_OK ){ JsonEachCursor *pCur = (JsonEachCursor*)*ppCursor; pCur->bRecursive = 1; } return rc; } /* Reset a JsonEachCursor back to its original state. Free any memory ** held. */ static void jsonEachCursorReset(JsonEachCursor *p){ jsonParseReset(&p->sParse); jsonStringReset(&p->path); sqlite3DbFree(p->db, p->aParent); p->iRowid = 0; p->i = 0; p->aParent = 0; p->nParent = 0; p->nParentAlloc = 0; p->iEnd = 0; p->eType = 0; } /* Destructor for a jsonEachCursor object */ static int jsonEachClose(sqlite3_vtab_cursor *cur){ JsonEachCursor *p = (JsonEachCursor*)cur; jsonEachCursorReset(p); sqlite3DbFree(p->db, cur); return SQLITE_OK; } /* Return TRUE if the jsonEachCursor object has been advanced off the end ** of the JSON object */ static int jsonEachEof(sqlite3_vtab_cursor *cur){ JsonEachCursor *p = (JsonEachCursor*)cur; return p->i >= p->iEnd; } /* ** If the cursor is currently pointing at the label of a object entry, ** then return the index of the value. For all other cases, return the ** current pointer position, which is the value. */ static int jsonSkipLabel(JsonEachCursor *p){ if( p->eType==JSONB_OBJECT ){ u32 sz = 0; u32 n = jsonbPayloadSize(&p->sParse, p->i, &sz); return p->i + n + sz; }else{ return p->i; } } /* ** Append the path name for the current element. */ static void jsonAppendPathName(JsonEachCursor *p){ assert( p->nParent>0 ); assert( p->eType==JSONB_ARRAY || p->eType==JSONB_OBJECT ); if( p->eType==JSONB_ARRAY ){ jsonPrintf(30, &p->path, "[%lld]", p->aParent[p->nParent-1].iKey); }else{ u32 n, sz = 0, k, i; const char *z; int needQuote = 0; n = jsonbPayloadSize(&p->sParse, p->i, &sz); k = p->i + n; z = (const char*)&p->sParse.aBlob[k]; if( sz==0 || !sqlite3Isalpha(z[0]) ){ needQuote = 1; }else{ for(i=0; i<sz; i++){ if( !sqlite3Isalnum(z[i]) ){ needQuote = 1; break; } } } if( needQuote ){ jsonPrintf(sz+4,&p->path,".\"%.*s\"", sz, z); }else{ jsonPrintf(sz+2,&p->path,".%.*s", sz, z); } } } /* Advance the cursor to the next element for json_tree() */ static int jsonEachNext(sqlite3_vtab_cursor *cur){ JsonEachCursor *p = (JsonEachCursor*)cur; int rc = SQLITE_OK; if( p->bRecursive ){ u8 x; u8 levelChange = 0; u32 n, sz = 0; u32 i = jsonSkipLabel(p); x = p->sParse.aBlob[i] & 0x0f; n = jsonbPayloadSize(&p->sParse, i, &sz); if( x==JSONB_OBJECT || x==JSONB_ARRAY ){ JsonParent *pParent; if( p->nParent>=p->nParentAlloc ){ JsonParent *pNew; u64 nNew; nNew = p->nParentAlloc*2 + 3; pNew = sqlite3DbRealloc(p->db, p->aParent, sizeof(JsonParent)*nNew); if( pNew==0 ) return SQLITE_NOMEM; p->nParentAlloc = (u32)nNew; p->aParent = pNew; } levelChange = 1; pParent = &p->aParent[p->nParent]; pParent->iHead = p->i; pParent->iValue = i; pParent->iEnd = i + n + sz; pParent->iKey = -1; pParent->nPath = (u32)p->path.nUsed; if( p->eType && p->nParent ){ jsonAppendPathName(p); if( p->path.eErr ) rc = SQLITE_NOMEM; } p->nParent++; p->i = i + n; }else{ p->i = i + n + sz; } while( p->nParent>0 && p->i >= p->aParent[p->nParent-1].iEnd ){ p->nParent--; p->path.nUsed = p->aParent[p->nParent].nPath; levelChange = 1; } if( levelChange ){ if( p->nParent>0 ){ JsonParent *pParent = &p->aParent[p->nParent-1]; u32 iVal = pParent->iValue; p->eType = p->sParse.aBlob[iVal] & 0x0f; }else{ p->eType = 0; } } }else{ u32 n, sz = 0; u32 i = jsonSkipLabel(p); n = jsonbPayloadSize(&p->sParse, i, &sz); p->i = i + n + sz; } if( p->eType==JSONB_ARRAY && p->nParent ){ p->aParent[p->nParent-1].iKey++; } p->iRowid++; return rc; } /* Length of the path for rowid==0 in bRecursive mode. */ static int jsonEachPathLength(JsonEachCursor *p){ u32 n = p->path.nUsed; char *z = p->path.zBuf; if( p->iRowid==0 && p->bRecursive && n>=2 ){ while( n>1 ){ n--; if( z[n]=='[' || z[n]=='.' ){ u32 x, sz = 0; char cSaved = z[n]; z[n] = 0; assert( p->sParse.eEdit==0 ); x = jsonLookupStep(&p->sParse, 0, z+1, 0); z[n] = cSaved; if( JSON_LOOKUP_ISERROR(x) ) continue; if( x + jsonbPayloadSize(&p->sParse, x, &sz) == p->i ) break; } } } return n; } /* Return the value of a column */ static int jsonEachColumn( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int iColumn /* Which column to return */ ){ JsonEachCursor *p = (JsonEachCursor*)cur; switch( iColumn ){ case JEACH_KEY: { if( p->nParent==0 ){ u32 n, j; if( p->nRoot==1 ) break; j = jsonEachPathLength(p); n = p->nRoot - j; if( n==0 ){ break; }else if( p->path.zBuf[j]=='[' ){ i64 x; sqlite3Atoi64(&p->path.zBuf[j+1], &x, n-1, SQLITE_UTF8); sqlite3_result_int64(ctx, x); }else if( p->path.zBuf[j+1]=='"' ){ sqlite3_result_text(ctx, &p->path.zBuf[j+2], n-3, SQLITE_TRANSIENT); }else{ sqlite3_result_text(ctx, &p->path.zBuf[j+1], n-1, SQLITE_TRANSIENT); } break; } if( p->eType==JSONB_OBJECT ){ jsonReturnFromBlob(&p->sParse, p->i, ctx, 1); }else{ assert( p->eType==JSONB_ARRAY ); sqlite3_result_int64(ctx, p->aParent[p->nParent-1].iKey); } break; } case JEACH_VALUE: { u32 i = jsonSkipLabel(p); jsonReturnFromBlob(&p->sParse, i, ctx, 1); break; } case JEACH_TYPE: { u32 i = jsonSkipLabel(p); u8 eType = p->sParse.aBlob[i] & 0x0f; sqlite3_result_text(ctx, jsonbType[eType], -1, SQLITE_STATIC); break; } case JEACH_ATOM: { u32 i = jsonSkipLabel(p); if( (p->sParse.aBlob[i] & 0x0f)<JSONB_ARRAY ){ jsonReturnFromBlob(&p->sParse, i, ctx, 1); } break; } case JEACH_ID: { sqlite3_result_int64(ctx, (sqlite3_int64)p->i); break; } case JEACH_PARENT: { if( p->nParent>0 && p->bRecursive ){ sqlite3_result_int64(ctx, p->aParent[p->nParent-1].iHead); } break; } case JEACH_FULLKEY: { u64 nBase = p->path.nUsed; if( p->nParent ) jsonAppendPathName(p); sqlite3_result_text64(ctx, p->path.zBuf, p->path.nUsed, SQLITE_TRANSIENT, SQLITE_UTF8); p->path.nUsed = nBase; break; } case JEACH_PATH: { u32 n = jsonEachPathLength(p); sqlite3_result_text64(ctx, p->path.zBuf, n, SQLITE_TRANSIENT, SQLITE_UTF8); break; } default: { sqlite3_result_text(ctx, p->path.zBuf, p->nRoot, SQLITE_STATIC); break; } case JEACH_JSON: { if( p->sParse.zJson==0 ){ sqlite3_result_blob(ctx, p->sParse.aBlob, p->sParse.nBlob, SQLITE_STATIC); }else{ sqlite3_result_text(ctx, p->sParse.zJson, -1, SQLITE_STATIC); } break; } } return SQLITE_OK; } /* Return the current rowid value */ |
︙ | ︙ | |||
206177 206178 206179 206180 206181 206182 206183 | /* Start a search on a new JSON string */ static int jsonEachFilter( sqlite3_vtab_cursor *cur, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ JsonEachCursor *p = (JsonEachCursor*)cur; | < | < < > | > | > | > > > > | < | | > | < < | < | < < | | | < < < < < < > > | < | | > | | > | > | | | > | | > > > > > > | < | < < > > > | | > | < < < < < < < < < < < | > | | > > > | | > | > > > > > > > > > > > | > | > > > > | | 208169 208170 208171 208172 208173 208174 208175 208176 208177 208178 208179 208180 208181 208182 208183 208184 208185 208186 208187 208188 208189 208190 208191 208192 208193 208194 208195 208196 208197 208198 208199 208200 208201 208202 208203 208204 208205 208206 208207 208208 208209 208210 208211 208212 208213 208214 208215 208216 208217 208218 208219 208220 208221 208222 208223 208224 208225 208226 208227 208228 208229 208230 208231 208232 208233 208234 208235 208236 208237 208238 208239 208240 208241 208242 208243 208244 208245 208246 208247 208248 208249 208250 208251 208252 208253 208254 208255 208256 208257 208258 208259 208260 208261 208262 208263 208264 208265 208266 208267 208268 208269 208270 208271 208272 208273 208274 208275 208276 208277 | /* Start a search on a new JSON string */ static int jsonEachFilter( sqlite3_vtab_cursor *cur, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ JsonEachCursor *p = (JsonEachCursor*)cur; const char *zRoot = 0; u32 i, n, sz; UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(argc); jsonEachCursorReset(p); if( idxNum==0 ) return SQLITE_OK; memset(&p->sParse, 0, sizeof(p->sParse)); p->sParse.nJPRef = 1; p->sParse.db = p->db; if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){ if( jsonFuncArgMightBeBinary(argv[0]) ){ p->sParse.nBlob = sqlite3_value_bytes(argv[0]); p->sParse.aBlob = (u8*)sqlite3_value_blob(argv[0]); }else{ goto json_each_malformed_input; } }else{ p->sParse.zJson = (char*)sqlite3_value_text(argv[0]); p->sParse.nJson = sqlite3_value_bytes(argv[0]); if( p->sParse.zJson==0 ){ p->i = p->iEnd = 0; return SQLITE_OK; } if( jsonConvertTextToBlob(&p->sParse, 0) ){ if( p->sParse.oom ){ return SQLITE_NOMEM; } goto json_each_malformed_input; } } if( idxNum==3 ){ zRoot = (const char*)sqlite3_value_text(argv[1]); if( zRoot==0 ) return SQLITE_OK; if( zRoot[0]!='$' ){ sqlite3_free(cur->pVtab->zErrMsg); cur->pVtab->zErrMsg = jsonBadPathError(0, zRoot); jsonEachCursorReset(p); return cur->pVtab->zErrMsg ? SQLITE_ERROR : SQLITE_NOMEM; } p->nRoot = sqlite3Strlen30(zRoot); if( zRoot[1]==0 ){ i = p->i = 0; p->eType = 0; }else{ i = jsonLookupStep(&p->sParse, 0, zRoot+1, 0); if( JSON_LOOKUP_ISERROR(i) ){ if( i==JSON_LOOKUP_NOTFOUND ){ p->i = 0; p->eType = 0; p->iEnd = 0; return SQLITE_OK; } sqlite3_free(cur->pVtab->zErrMsg); cur->pVtab->zErrMsg = jsonBadPathError(0, zRoot); jsonEachCursorReset(p); return cur->pVtab->zErrMsg ? SQLITE_ERROR : SQLITE_NOMEM; } if( p->sParse.iLabel ){ p->i = p->sParse.iLabel; p->eType = JSONB_OBJECT; }else{ p->i = i; p->eType = JSONB_ARRAY; } } jsonAppendRaw(&p->path, zRoot, p->nRoot); }else{ i = p->i = 0; p->eType = 0; p->nRoot = 1; jsonAppendRaw(&p->path, "$", 1); } p->nParent = 0; n = jsonbPayloadSize(&p->sParse, i, &sz); p->iEnd = i+n+sz; if( (p->sParse.aBlob[i] & 0x0f)>=JSONB_ARRAY && !p->bRecursive ){ p->i = i + n; p->eType = p->sParse.aBlob[i] & 0x0f; p->aParent = sqlite3DbMallocZero(p->db, sizeof(JsonParent)); if( p->aParent==0 ) return SQLITE_NOMEM; p->nParent = 1; p->nParentAlloc = 1; p->aParent[0].iKey = 0; p->aParent[0].iEnd = p->iEnd; p->aParent[0].iHead = p->i; p->aParent[0].iValue = i; } return SQLITE_OK; json_each_malformed_input: sqlite3_free(cur->pVtab->zErrMsg); cur->pVtab->zErrMsg = sqlite3_mprintf("malformed JSON"); jsonEachCursorReset(p); return cur->pVtab->zErrMsg ? SQLITE_ERROR : SQLITE_NOMEM; } /* The methods of the json_each virtual table */ static sqlite3_module jsonEachModule = { 0, /* iVersion */ 0, /* xCreate */ jsonEachConnect, /* xConnect */ |
︙ | ︙ | |||
206325 206326 206327 206328 206329 206330 206331 | /* ** Register JSON functions. */ SQLITE_PRIVATE void sqlite3RegisterJsonFunctions(void){ #ifndef SQLITE_OMIT_JSON static FuncDef aJsonFunc[] = { | > > > > > > | > | > | | | | > | | | > | > | > | | > | > | > | | | > | < > | > > > > > > > | | 208332 208333 208334 208335 208336 208337 208338 208339 208340 208341 208342 208343 208344 208345 208346 208347 208348 208349 208350 208351 208352 208353 208354 208355 208356 208357 208358 208359 208360 208361 208362 208363 208364 208365 208366 208367 208368 208369 208370 208371 208372 208373 208374 208375 208376 208377 208378 208379 208380 208381 208382 208383 208384 208385 208386 208387 208388 208389 208390 208391 208392 208393 208394 208395 208396 | /* ** Register JSON functions. */ SQLITE_PRIVATE void sqlite3RegisterJsonFunctions(void){ #ifndef SQLITE_OMIT_JSON static FuncDef aJsonFunc[] = { /* sqlite3_result_subtype() ----, ,--- sqlite3_value_subtype() */ /* | | */ /* Uses cache ------, | | ,---- Returns JSONB */ /* | | | | */ /* Number of arguments ---, | | | | ,--- Flags */ /* | | | | | | */ JFUNCTION(json, 1,1,1, 0,0,0, jsonRemoveFunc), JFUNCTION(jsonb, 1,1,0, 0,1,0, jsonRemoveFunc), JFUNCTION(json_array, -1,0,1, 1,0,0, jsonArrayFunc), JFUNCTION(jsonb_array, -1,0,1, 1,1,0, jsonArrayFunc), JFUNCTION(json_array_length, 1,1,0, 0,0,0, jsonArrayLengthFunc), JFUNCTION(json_array_length, 2,1,0, 0,0,0, jsonArrayLengthFunc), JFUNCTION(json_error_position,1,1,0, 0,0,0, jsonErrorFunc), JFUNCTION(json_extract, -1,1,1, 0,0,0, jsonExtractFunc), JFUNCTION(jsonb_extract, -1,1,0, 0,1,0, jsonExtractFunc), JFUNCTION(->, 2,1,1, 0,0,JSON_JSON, jsonExtractFunc), JFUNCTION(->>, 2,1,0, 0,0,JSON_SQL, jsonExtractFunc), JFUNCTION(json_insert, -1,1,1, 1,0,0, jsonSetFunc), JFUNCTION(jsonb_insert, -1,1,0, 1,1,0, jsonSetFunc), JFUNCTION(json_object, -1,0,1, 1,0,0, jsonObjectFunc), JFUNCTION(jsonb_object, -1,0,1, 1,1,0, jsonObjectFunc), JFUNCTION(json_patch, 2,1,1, 0,0,0, jsonPatchFunc), JFUNCTION(jsonb_patch, 2,1,0, 0,1,0, jsonPatchFunc), JFUNCTION(json_quote, 1,0,1, 1,0,0, jsonQuoteFunc), JFUNCTION(json_remove, -1,1,1, 0,0,0, jsonRemoveFunc), JFUNCTION(jsonb_remove, -1,1,0, 0,1,0, jsonRemoveFunc), JFUNCTION(json_replace, -1,1,1, 1,0,0, jsonReplaceFunc), JFUNCTION(jsonb_replace, -1,1,0, 1,1,0, jsonReplaceFunc), JFUNCTION(json_set, -1,1,1, 1,0,JSON_ISSET, jsonSetFunc), JFUNCTION(jsonb_set, -1,1,0, 1,1,JSON_ISSET, jsonSetFunc), JFUNCTION(json_type, 1,1,0, 0,0,0, jsonTypeFunc), JFUNCTION(json_type, 2,1,0, 0,0,0, jsonTypeFunc), JFUNCTION(json_valid, 1,1,0, 0,0,0, jsonValidFunc), JFUNCTION(json_valid, 2,1,0, 0,0,0, jsonValidFunc), #if SQLITE_DEBUG JFUNCTION(json_parse, 1,1,0, 0,0,0, jsonParseFunc), #endif WAGGREGATE(json_group_array, 1, 0, 0, jsonArrayStep, jsonArrayFinal, jsonArrayValue, jsonGroupInverse, SQLITE_SUBTYPE|SQLITE_RESULT_SUBTYPE|SQLITE_UTF8| SQLITE_DETERMINISTIC), WAGGREGATE(jsonb_group_array, 1, JSON_BLOB, 0, jsonArrayStep, jsonArrayFinal, jsonArrayValue, jsonGroupInverse, SQLITE_SUBTYPE|SQLITE_RESULT_SUBTYPE|SQLITE_UTF8|SQLITE_DETERMINISTIC), WAGGREGATE(json_group_object, 2, 0, 0, jsonObjectStep, jsonObjectFinal, jsonObjectValue, jsonGroupInverse, SQLITE_SUBTYPE|SQLITE_RESULT_SUBTYPE|SQLITE_UTF8|SQLITE_DETERMINISTIC), WAGGREGATE(jsonb_group_object,2, JSON_BLOB, 0, jsonObjectStep, jsonObjectFinal, jsonObjectValue, jsonGroupInverse, SQLITE_SUBTYPE|SQLITE_RESULT_SUBTYPE|SQLITE_UTF8| SQLITE_DETERMINISTIC) }; sqlite3InsertBuiltinFuncs(aJsonFunc, ArraySize(aJsonFunc)); #endif } #if !defined(SQLITE_OMIT_VIRTUALTABLE) && !defined(SQLITE_OMIT_JSON) /* |
︙ | ︙ | |||
207100 207101 207102 207103 207104 207105 207106 | int rc = SQLITE_OK; RtreeNode *pNode = 0; /* Check if the requested node is already in the hash table. If so, ** increase its reference count and return it. */ if( (pNode = nodeHashLookup(pRtree, iNode))!=0 ){ | | | 209130 209131 209132 209133 209134 209135 209136 209137 209138 209139 209140 209141 209142 209143 209144 | int rc = SQLITE_OK; RtreeNode *pNode = 0; /* Check if the requested node is already in the hash table. If so, ** increase its reference count and return it. */ if( (pNode = nodeHashLookup(pRtree, iNode))!=0 ){ if( pParent && ALWAYS(pParent!=pNode->pParent) ){ RTREE_IS_CORRUPT(pRtree); return SQLITE_CORRUPT_VTAB; } pNode->nRef++; *ppNode = pNode; return SQLITE_OK; } |
︙ | ︙ | |||
209715 209716 209717 209718 209719 209720 209721 | for(i=0; i<sizeof(azName)/sizeof(azName[0]); i++){ if( sqlite3_stricmp(zName, azName[i])==0 ) return 1; } return 0; } /* Forward declaration */ | | | 211745 211746 211747 211748 211749 211750 211751 211752 211753 211754 211755 211756 211757 211758 211759 | for(i=0; i<sizeof(azName)/sizeof(azName[0]); i++){ if( sqlite3_stricmp(zName, azName[i])==0 ) return 1; } return 0; } /* Forward declaration */ static int rtreeIntegrity(sqlite3_vtab*, const char*, const char*, int, char**); static sqlite3_module rtreeModule = { 4, /* iVersion */ rtreeCreate, /* xCreate - create a table */ rtreeConnect, /* xConnect - connect to an existing table */ rtreeBestIndex, /* xBestIndex - Determine search strategy */ rtreeDisconnect, /* xDisconnect - Disconnect from a table */ |
︙ | ︙ | |||
209835 209836 209837 209838 209839 209840 209841 | if( zSql ){ rc = sqlite3_prepare_v3(db, zSql, -1, f, appStmt[i], 0); }else{ rc = SQLITE_NOMEM; } sqlite3_free(zSql); } | | | 211865 211866 211867 211868 211869 211870 211871 211872 211873 211874 211875 211876 211877 211878 211879 | if( zSql ){ rc = sqlite3_prepare_v3(db, zSql, -1, f, appStmt[i], 0); }else{ rc = SQLITE_NOMEM; } sqlite3_free(zSql); } if( pRtree->nAux && rc!=SQLITE_NOMEM ){ pRtree->zReadAuxSql = sqlite3_mprintf( "SELECT * FROM \"%w\".\"%w_rowid\" WHERE rowid=?1", zDb, zPrefix); if( pRtree->zReadAuxSql==0 ){ rc = SQLITE_NOMEM; }else{ sqlite3_str *p = sqlite3_str_new(db); |
︙ | ︙ | |||
210524 210525 210526 210527 210528 210529 210530 | /* Initialize the context object */ memset(&check, 0, sizeof(check)); check.db = db; check.zDb = zDb; check.zTab = zTab; /* Find the number of auxiliary columns */ | < | | | | | | | < | 212554 212555 212556 212557 212558 212559 212560 212561 212562 212563 212564 212565 212566 212567 212568 212569 212570 212571 212572 212573 212574 | /* Initialize the context object */ memset(&check, 0, sizeof(check)); check.db = db; check.zDb = zDb; check.zTab = zTab; /* Find the number of auxiliary columns */ pStmt = rtreeCheckPrepare(&check, "SELECT * FROM %Q.'%q_rowid'", zDb, zTab); if( pStmt ){ nAux = sqlite3_column_count(pStmt) - 2; sqlite3_finalize(pStmt); }else if( check.rc!=SQLITE_NOMEM ){ check.rc = SQLITE_OK; } /* Find number of dimensions in the rtree table. */ pStmt = rtreeCheckPrepare(&check, "SELECT * FROM %Q.%Q", zDb, zTab); if( pStmt ){ int rc; check.nDim = (sqlite3_column_count(pStmt) - 1 - nAux) / 2; |
︙ | ︙ | |||
210570 210571 210572 210573 210574 210575 210576 | *pzReport = check.zReport; return check.rc; } /* ** Implementation of the xIntegrity method for Rtree. */ | | > > > > > > > > > > > | 212598 212599 212600 212601 212602 212603 212604 212605 212606 212607 212608 212609 212610 212611 212612 212613 212614 212615 212616 212617 212618 212619 212620 212621 212622 212623 212624 212625 212626 212627 212628 212629 | *pzReport = check.zReport; return check.rc; } /* ** Implementation of the xIntegrity method for Rtree. */ static int rtreeIntegrity( sqlite3_vtab *pVtab, /* The virtual table to check */ const char *zSchema, /* Schema in which the virtual table lives */ const char *zName, /* Name of the virtual table */ int isQuick, /* True for a quick_check */ char **pzErr /* Write results here */ ){ Rtree *pRtree = (Rtree*)pVtab; int rc; assert( pzErr!=0 && *pzErr==0 ); UNUSED_PARAMETER(zSchema); UNUSED_PARAMETER(zName); UNUSED_PARAMETER(isQuick); rc = rtreeCheckTable(pRtree->db, pRtree->zDb, pRtree->zName, pzErr); if( rc==SQLITE_OK && *pzErr ){ *pzErr = sqlite3_mprintf("In RTree %s.%s:\n%z", pRtree->zDb, pRtree->zName, *pzErr); if( (*pzErr)==0 ) rc = SQLITE_NOMEM; } return rc; } /* ** Usage: ** |
︙ | ︙ | |||
222167 222168 222169 222170 222171 222172 222173 | pSession->nMaxChangesetSize += (nCol - nOldCol); pSession->nMaxChangesetSize += sessionVarintLen(nCol); pSession->nMaxChangesetSize -= sessionVarintLen(nOldCol); } } } | | | 224206 224207 224208 224209 224210 224211 224212 224213 224214 224215 224216 224217 224218 224219 224220 | pSession->nMaxChangesetSize += (nCol - nOldCol); pSession->nMaxChangesetSize += sessionVarintLen(nCol); pSession->nMaxChangesetSize -= sessionVarintLen(nOldCol); } } } sqlite3_free((char*)azCol); return pSession->rc; } /* ** Session-change object (*pp) contains an old.* record with fewer than ** nCol fields. This function updates it with the default values for ** the missing fields. |
︙ | ︙ | |||
223247 223248 223249 223250 223251 223252 223253 | sqlite3_mutex_leave(sqlite3_db_mutex(db)); sqlite3ValueFree(pSession->pZeroBlob); /* Delete all attached table objects. And the contents of their ** associated hash-tables. */ sessionDeleteTable(pSession, pSession->pTable); | < | < | 225286 225287 225288 225289 225290 225291 225292 225293 225294 225295 225296 225297 225298 225299 225300 | sqlite3_mutex_leave(sqlite3_db_mutex(db)); sqlite3ValueFree(pSession->pZeroBlob); /* Delete all attached table objects. And the contents of their ** associated hash-tables. */ sessionDeleteTable(pSession, pSession->pTable); /* Free the session object. */ sqlite3_free(pSession); } /* ** Set a table filter on a Session Object. */ SQLITE_API void sqlite3session_table_filter( |
︙ | ︙ | |||
227462 227463 227464 227465 227466 227467 227468 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: | > > > | | > > | | | > | | | | 229499 229500 229501 229502 229503 229504 229505 229506 229507 229508 229509 229510 229511 229512 229513 229514 229515 229516 229517 229518 229519 229520 229521 229522 229523 229524 229525 229526 229527 229528 229529 229530 229531 229532 229533 229534 229535 229536 229537 229538 229539 229540 229541 229542 229543 229544 229545 229546 229547 229548 229549 229550 229551 229552 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: ** If parameter iCol is less than zero, or greater than or equal to the ** number of columns in the table, SQLITE_RANGE is returned. ** ** Otherwise, this function attempts to retrieve the text of column iCol of ** the current document. If successful, (*pz) is set to point to a buffer ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes ** (not characters) of the buffer and SQLITE_OK is returned. Otherwise, ** if an error occurs, an SQLite error code is returned and the final values ** of (*pz) and (*pn) are undefined. ** ** xPhraseCount: ** Returns the number of phrases in the current query expression. ** ** xPhraseSize: ** If parameter iCol is less than zero, or greater than or equal to the ** number of phrases in the current query, as returned by xPhraseCount, ** 0 is returned. Otherwise, this function returns the number of tokens in ** phrase iPhrase of the query. Phrases are numbered starting from zero. ** ** xInstCount: ** Set *pnInst to the total number of occurrences of all phrases within ** the query within the current row. Return SQLITE_OK if successful, or ** an error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. If the FTS5 table is created ** with either "detail=none" or "detail=column" and "content=" option ** (i.e. if it is a contentless table), then this API always returns 0. ** ** xInst: ** Query for the details of phrase match iIdx within the current row. ** Phrase matches are numbered starting from zero, so the iIdx argument ** should be greater than or equal to zero and smaller than the value ** output by xInstCount(). If iIdx is less than zero or greater than ** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned. ** ** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol ** to the column in which it occurs and *piOff the token offset of the ** first token of the phrase. SQLITE_OK is returned if successful, or an ** error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. ** ** xRowid: ** Returns the rowid of the current row. ** |
︙ | ︙ | |||
227520 227521 227522 227523 227524 227525 227526 227527 227528 227529 227530 227531 227532 227533 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. | > > > > | 229563 229564 229565 229566 229567 229568 229569 229570 229571 229572 229573 229574 229575 229576 229577 229578 229579 229580 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If parameter iPhrase is less than zero, or greater than or equal to ** the number of phrases in the query, as returned by xPhraseCount(), ** this function returns SQLITE_RANGE. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. |
︙ | ︙ | |||
227635 227636 227637 227638 227639 227640 227641 227642 227643 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. */ struct Fts5ExtensionApi { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | 229682 229683 229684 229685 229686 229687 229688 229689 229690 229691 229692 229693 229694 229695 229696 229697 229698 229699 229700 229701 229702 229703 229704 229705 229706 229707 229708 229709 229710 229711 229712 229713 229714 229715 229716 229717 229718 229719 229720 229721 229722 229723 229724 229725 229726 229727 229728 229729 229730 229731 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. ** ** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase iPhrase of the current ** query. Before returning, output parameter *ppToken is set to point ** to a buffer containing the requested token, and *pnToken to the ** size of this buffer in bytes. ** ** If iPhrase or iToken are less than zero, or if iPhrase is greater than ** or equal to the number of phrases in the query as reported by ** xPhraseCount(), or if iToken is equal to or greater than the number of ** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken are both zeroed. ** ** The output text is not a copy of the query text that specified the ** token. It is the output of the tokenizer module. For tokendata=1 ** tables, this includes any embedded 0x00 and trailing data. ** ** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase hit iIdx within the ** current row. If iIdx is less than zero or greater than or equal to the ** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise, ** output variable (*ppToken) is set to point to a buffer containing the ** matching document token, and (*pnToken) to the size of that buffer in ** bytes. This API is not available if the specified token matches a ** prefix query term. In that case both output variables are always set ** to 0. ** ** The output text is not a copy of the document text that was tokenized. ** It is the output of the tokenizer module. For tokendata=1 tables, this ** includes any embedded 0x00 and trailing data. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. */ struct Fts5ExtensionApi { int iVersion; /* Currently always set to 3 */ void *(*xUserData)(Fts5Context*); int (*xColumnCount)(Fts5Context*); int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow); int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken); |
︙ | ︙ | |||
227672 227673 227674 227675 227676 227677 227678 227679 227680 227681 227682 227683 227684 227685 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* | > > > > > > > | 229752 229753 229754 229755 229756 229757 229758 229759 229760 229761 229762 229763 229764 229765 229766 229767 229768 229769 229770 229771 229772 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); /* Below this point are iVersion>=3 only */ int (*xQueryToken)(Fts5Context*, int iPhrase, int iToken, const char **ppToken, int *pnToken ); int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* |
︙ | ︙ | |||
228146 228147 228148 228149 228150 228151 228152 228153 228154 228155 228156 228157 228158 228159 | int nPrefix; /* Number of prefix indexes */ int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */ int eContent; /* An FTS5_CONTENT value */ int bContentlessDelete; /* "contentless_delete=" option (dflt==0) */ char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; Fts5Tokenizer *pTok; fts5_tokenizer *pTokApi; int bLock; /* True when table is preparing statement */ int ePattern; /* FTS_PATTERN_XXX constant */ | > | 230233 230234 230235 230236 230237 230238 230239 230240 230241 230242 230243 230244 230245 230246 230247 | int nPrefix; /* Number of prefix indexes */ int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */ int eContent; /* An FTS5_CONTENT value */ int bContentlessDelete; /* "contentless_delete=" option (dflt==0) */ char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int bTokendata; /* "tokendata=" option value (dflt==0) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; Fts5Tokenizer *pTok; fts5_tokenizer *pTokApi; int bLock; /* True when table is preparing statement */ int ePattern; /* FTS_PATTERN_XXX constant */ |
︙ | ︙ | |||
228334 228335 228336 228337 228338 228339 228340 | }; #define sqlite3Fts5IterEof(x) ((x)->bEof) /* ** Values used as part of the flags argument passed to IndexQuery(). */ | | | | | | | | > > | 230422 230423 230424 230425 230426 230427 230428 230429 230430 230431 230432 230433 230434 230435 230436 230437 230438 230439 230440 230441 230442 230443 230444 230445 230446 230447 230448 | }; #define sqlite3Fts5IterEof(x) ((x)->bEof) /* ** Values used as part of the flags argument passed to IndexQuery(). */ #define FTS5INDEX_QUERY_PREFIX 0x0001 /* Prefix query */ #define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */ #define FTS5INDEX_QUERY_TEST_NOIDX 0x0004 /* Do not use prefix index */ #define FTS5INDEX_QUERY_SCAN 0x0008 /* Scan query (fts5vocab) */ /* The following are used internally by the fts5_index.c module. They are ** defined here only to make it easier to avoid clashes with the flags ** above. */ #define FTS5INDEX_QUERY_SKIPEMPTY 0x0010 #define FTS5INDEX_QUERY_NOOUTPUT 0x0020 #define FTS5INDEX_QUERY_SKIPHASH 0x0040 #define FTS5INDEX_QUERY_NOTOKENDATA 0x0080 #define FTS5INDEX_QUERY_SCANONETERM 0x0100 /* ** Create/destroy an Fts5Index object. */ static int sqlite3Fts5IndexOpen(Fts5Config *pConfig, int bCreate, Fts5Index**, char**); static int sqlite3Fts5IndexClose(Fts5Index *p); |
︙ | ︙ | |||
228413 228414 228415 228416 228417 228418 228419 228420 228421 228422 228423 228424 228425 228426 | */ static const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); static int sqlite3Fts5IterNextScan(Fts5IndexIter*); static void *sqlite3Fts5StructureRef(Fts5Index*); static void sqlite3Fts5StructureRelease(void*); static int sqlite3Fts5StructureTest(Fts5Index*, void*); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** ** For an insert, it must be called once for each token in the new document. | > > > > | 230503 230504 230505 230506 230507 230508 230509 230510 230511 230512 230513 230514 230515 230516 230517 230518 230519 230520 | */ static const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); static int sqlite3Fts5IterNextScan(Fts5IndexIter*); static void *sqlite3Fts5StructureRef(Fts5Index*); static void sqlite3Fts5StructureRelease(void*); static int sqlite3Fts5StructureTest(Fts5Index*, void*); /* ** Used by xInstToken(): */ static int sqlite3Fts5IterToken(Fts5IndexIter*, i64, int, int, const char**, int*); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** ** For an insert, it must be called once for each token in the new document. |
︙ | ︙ | |||
228489 228490 228491 228492 228493 228494 228495 228496 228497 228498 228499 228500 228501 228502 | static int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge); static int sqlite3Fts5IndexReset(Fts5Index *p); static int sqlite3Fts5IndexLoadConfig(Fts5Index *p); static int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin); static int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid); /* ** End of interface to code in fts5_index.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_varint.c. | > > > > > > > | 230583 230584 230585 230586 230587 230588 230589 230590 230591 230592 230593 230594 230595 230596 230597 230598 230599 230600 230601 230602 230603 | static int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge); static int sqlite3Fts5IndexReset(Fts5Index *p); static int sqlite3Fts5IndexLoadConfig(Fts5Index *p); static int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin); static int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid); static void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter*); /* Used to populate hash tables for xInstToken in detail=none/column mode. */ static int sqlite3Fts5IndexIterWriteTokendata( Fts5IndexIter*, const char*, int, i64 iRowid, int iCol, int iOff ); /* ** End of interface to code in fts5_index.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_varint.c. |
︙ | ︙ | |||
228595 228596 228597 228598 228599 228600 228601 228602 228603 228604 228605 228606 228607 228608 | Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ); static void sqlite3Fts5HashScanNext(Fts5Hash*); static int sqlite3Fts5HashScanEof(Fts5Hash*); static void sqlite3Fts5HashScanEntry(Fts5Hash *, const char **pzTerm, /* OUT: term (nul-terminated) */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ); /* | > | 230696 230697 230698 230699 230700 230701 230702 230703 230704 230705 230706 230707 230708 230709 230710 | Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ); static void sqlite3Fts5HashScanNext(Fts5Hash*); static int sqlite3Fts5HashScanEof(Fts5Hash*); static void sqlite3Fts5HashScanEntry(Fts5Hash *, const char **pzTerm, /* OUT: term (nul-terminated) */ int *pnTerm, /* OUT: Size of term in bytes */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ); /* |
︙ | ︙ | |||
228720 228721 228722 228723 228724 228725 228726 228727 228728 228729 228730 228731 228732 228733 | Fts5Config*, Fts5Expr*, Fts5PoslistPopulator*, int, const char*, int ); static void sqlite3Fts5ExprCheckPoslists(Fts5Expr*, i64); static int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**); static int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *); /******************************************* ** The fts5_expr.c API above this point is used by the other hand-written ** C code in this module. The interfaces below this point are called by ** the parser code in fts5parse.y. */ static void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...); | > > > > | 230822 230823 230824 230825 230826 230827 230828 230829 230830 230831 230832 230833 230834 230835 230836 230837 230838 230839 | Fts5Config*, Fts5Expr*, Fts5PoslistPopulator*, int, const char*, int ); static void sqlite3Fts5ExprCheckPoslists(Fts5Expr*, i64); static int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**); static int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *); static int sqlite3Fts5ExprQueryToken(Fts5Expr*, int, int, const char**, int*); static int sqlite3Fts5ExprInstToken(Fts5Expr*, i64, int, int, int, int, const char**, int*); static void sqlite3Fts5ExprClearTokens(Fts5Expr*); /******************************************* ** The fts5_expr.c API above this point is used by the other hand-written ** C code in this module. The interfaces below this point are called by ** the parser code in fts5parse.y. */ static void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...); |
︙ | ︙ | |||
230435 230436 230437 230438 230439 230440 230441 | /************************************************************************* ** Start of highlight() implementation. */ typedef struct HighlightContext HighlightContext; struct HighlightContext { | | < > > > > | > | 232541 232542 232543 232544 232545 232546 232547 232548 232549 232550 232551 232552 232553 232554 232555 232556 232557 232558 232559 232560 232561 232562 232563 232564 232565 232566 232567 | /************************************************************************* ** Start of highlight() implementation. */ typedef struct HighlightContext HighlightContext; struct HighlightContext { /* Constant parameters to fts5HighlightCb() */ int iRangeStart; /* First token to include */ int iRangeEnd; /* If non-zero, last token to include */ const char *zOpen; /* Opening highlight */ const char *zClose; /* Closing highlight */ const char *zIn; /* Input text */ int nIn; /* Size of input text in bytes */ /* Variables modified by fts5HighlightCb() */ CInstIter iter; /* Coalesced Instance Iterator */ int iPos; /* Current token offset in zIn[] */ int iOff; /* Have copied up to this offset in zIn[] */ int bOpen; /* True if highlight is open */ char *zOut; /* Output value */ }; /* ** Append text to the HighlightContext output string - p->zOut. Argument ** z points to a buffer containing n bytes of text to append. If n is ** negative, everything up until the first '\0' is appended to the output. |
︙ | ︙ | |||
230476 230477 230478 230479 230480 230481 230482 | ** Tokenizer callback used by implementation of highlight() function. */ static int fts5HighlightCb( void *pContext, /* Pointer to HighlightContext object */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Buffer containing token */ int nToken, /* Size of token in bytes */ | | | > > > > > > > > > > > > > > > > > | > > | > < > | > > | | < > > > > | 232586 232587 232588 232589 232590 232591 232592 232593 232594 232595 232596 232597 232598 232599 232600 232601 232602 232603 232604 232605 232606 232607 232608 232609 232610 232611 232612 232613 232614 232615 232616 232617 232618 232619 232620 232621 232622 232623 232624 232625 232626 232627 232628 232629 232630 232631 232632 232633 232634 232635 232636 232637 232638 232639 232640 232641 232642 232643 232644 232645 232646 232647 232648 232649 232650 232651 232652 232653 232654 232655 232656 232657 232658 232659 232660 232661 232662 232663 232664 232665 | ** Tokenizer callback used by implementation of highlight() function. */ static int fts5HighlightCb( void *pContext, /* Pointer to HighlightContext object */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Buffer containing token */ int nToken, /* Size of token in bytes */ int iStartOff, /* Start byte offset of token */ int iEndOff /* End byte offset of token */ ){ HighlightContext *p = (HighlightContext*)pContext; int rc = SQLITE_OK; int iPos; UNUSED_PARAM2(pToken, nToken); if( tflags & FTS5_TOKEN_COLOCATED ) return SQLITE_OK; iPos = p->iPos++; if( p->iRangeEnd>=0 ){ if( iPos<p->iRangeStart || iPos>p->iRangeEnd ) return SQLITE_OK; if( p->iRangeStart && iPos==p->iRangeStart ) p->iOff = iStartOff; } /* If the parenthesis is open, and this token is not part of the current ** phrase, and the starting byte offset of this token is past the point ** that has currently been copied into the output buffer, close the ** parenthesis. */ if( p->bOpen && (iPos<=p->iter.iStart || p->iter.iStart<0) && iStartOff>p->iOff ){ fts5HighlightAppend(&rc, p, p->zClose, -1); p->bOpen = 0; } /* If this is the start of a new phrase, and the highlight is not open: ** ** * copy text from the input up to the start of the phrase, and ** * open the highlight. */ if( iPos==p->iter.iStart && p->bOpen==0 ){ fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iStartOff - p->iOff); fts5HighlightAppend(&rc, p, p->zOpen, -1); p->iOff = iStartOff; p->bOpen = 1; } if( iPos==p->iter.iEnd ){ if( p->bOpen==0 ){ assert( p->iRangeEnd>=0 ); fts5HighlightAppend(&rc, p, p->zOpen, -1); p->bOpen = 1; } fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; if( rc==SQLITE_OK ){ rc = fts5CInstIterNext(&p->iter); } } if( iPos==p->iRangeEnd ){ if( p->bOpen ){ if( p->iter.iStart>=0 && iPos>=p->iter.iStart ){ fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; } fts5HighlightAppend(&rc, p, p->zClose, -1); p->bOpen = 0; } fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; } return rc; } /* ** Implementation of highlight() function. |
︙ | ︙ | |||
230548 230549 230550 230551 230552 230553 230554 | iCol = sqlite3_value_int(apVal[0]); memset(&ctx, 0, sizeof(HighlightContext)); ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]); ctx.zClose = (const char*)sqlite3_value_text(apVal[2]); ctx.iRangeEnd = -1; rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn); | | > > | > > > | 232683 232684 232685 232686 232687 232688 232689 232690 232691 232692 232693 232694 232695 232696 232697 232698 232699 232700 232701 232702 232703 232704 232705 232706 232707 232708 232709 232710 | iCol = sqlite3_value_int(apVal[0]); memset(&ctx, 0, sizeof(HighlightContext)); ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]); ctx.zClose = (const char*)sqlite3_value_text(apVal[2]); ctx.iRangeEnd = -1; rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn); if( rc==SQLITE_RANGE ){ sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC); rc = SQLITE_OK; }else if( ctx.zIn ){ if( rc==SQLITE_OK ){ rc = fts5CInstIterInit(pApi, pFts, iCol, &ctx.iter); } if( rc==SQLITE_OK ){ rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb); } if( ctx.bOpen ){ fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1); } fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff); if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, (const char*)ctx.zOut, -1, SQLITE_TRANSIENT); } sqlite3_free(ctx.zOut); } |
︙ | ︙ | |||
230835 230836 230837 230838 230839 230840 230841 230842 230843 230844 230845 230846 230847 230848 | while( ctx.iter.iStart>=0 && ctx.iter.iStart<iBestStart && rc==SQLITE_OK ){ rc = fts5CInstIterNext(&ctx.iter); } if( rc==SQLITE_OK ){ rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb); } if( ctx.iRangeEnd>=(nColSize-1) ){ fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff); }else{ fts5HighlightAppend(&rc, &ctx, zEllips, -1); } } if( rc==SQLITE_OK ){ | > > > | 232975 232976 232977 232978 232979 232980 232981 232982 232983 232984 232985 232986 232987 232988 232989 232990 232991 | while( ctx.iter.iStart>=0 && ctx.iter.iStart<iBestStart && rc==SQLITE_OK ){ rc = fts5CInstIterNext(&ctx.iter); } if( rc==SQLITE_OK ){ rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb); } if( ctx.bOpen ){ fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1); } if( ctx.iRangeEnd>=(nColSize-1) ){ fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff); }else{ fts5HighlightAppend(&rc, &ctx, zEllips, -1); } } if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
231110 231111 231112 231113 231114 231115 231116 231117 231118 231119 231120 231121 231122 231123 | int *pRc, Fts5Buffer *pBuf, u32 nData, const u8 *pData ){ if( nData ){ if( fts5BufferGrow(pRc, pBuf, nData) ) return; memcpy(&pBuf->p[pBuf->n], pData, nData); pBuf->n += nData; } } /* ** Append the nul-terminated string zStr to the buffer pBuf. This function | > | 233253 233254 233255 233256 233257 233258 233259 233260 233261 233262 233263 233264 233265 233266 233267 | int *pRc, Fts5Buffer *pBuf, u32 nData, const u8 *pData ){ if( nData ){ if( fts5BufferGrow(pRc, pBuf, nData) ) return; assert( pBuf->p!=0 ); memcpy(&pBuf->p[pBuf->n], pData, nData); pBuf->n += nData; } } /* ** Append the nul-terminated string zStr to the buffer pBuf. This function |
︙ | ︙ | |||
231211 231212 231213 231214 231215 231216 231217 231218 231219 231220 231221 231222 231223 231224 231225 231226 231227 231228 231229 231230 231231 | static int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ){ int i = *pi; if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; u32 iVal; fts5FastGetVarint32(a, i, iVal); if( iVal<=1 ){ if( iVal==0 ){ *pi = i; return 0; } fts5FastGetVarint32(a, i, iVal); | > > | 233355 233356 233357 233358 233359 233360 233361 233362 233363 233364 233365 233366 233367 233368 233369 233370 233371 233372 233373 233374 233375 233376 233377 | static int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ){ int i = *pi; assert( a!=0 || i==0 ); if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; u32 iVal; assert( a!=0 ); fts5FastGetVarint32(a, i, iVal); if( iVal<=1 ){ if( iVal==0 ){ *pi = i; return 0; } fts5FastGetVarint32(a, i, iVal); |
︙ | ︙ | |||
231848 231849 231850 231851 231852 231853 231854 231855 231856 231857 231858 231859 231860 231861 | }; if( (rc = fts5ConfigSetEnum(aDetail, zArg, &pConfig->eDetail)) ){ *pzErr = sqlite3_mprintf("malformed detail=... directive"); } return rc; } *pzErr = sqlite3_mprintf("unrecognized option: \"%.*s\"", nCmd, zCmd); return SQLITE_ERROR; } /* ** Allocate an instance of the default tokenizer ("simple") at | > > > > > > > > > > | 233994 233995 233996 233997 233998 233999 234000 234001 234002 234003 234004 234005 234006 234007 234008 234009 234010 234011 234012 234013 234014 234015 234016 234017 | }; if( (rc = fts5ConfigSetEnum(aDetail, zArg, &pConfig->eDetail)) ){ *pzErr = sqlite3_mprintf("malformed detail=... directive"); } return rc; } if( sqlite3_strnicmp("tokendata", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed tokendata=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bTokendata = (zArg[0]=='1'); } return rc; } *pzErr = sqlite3_mprintf("unrecognized option: \"%.*s\"", nCmd, zCmd); return SQLITE_ERROR; } /* ** Allocate an instance of the default tokenizer ("simple") at |
︙ | ︙ | |||
232582 232583 232584 232585 232586 232587 232588 | /* ** An instance of the following structure represents a single search term ** or term prefix. */ struct Fts5ExprTerm { u8 bPrefix; /* True for a prefix term */ u8 bFirst; /* True if token must be first in column */ | | > > | 234738 234739 234740 234741 234742 234743 234744 234745 234746 234747 234748 234749 234750 234751 234752 234753 234754 | /* ** An instance of the following structure represents a single search term ** or term prefix. */ struct Fts5ExprTerm { u8 bPrefix; /* True for a prefix term */ u8 bFirst; /* True if token must be first in column */ char *pTerm; /* Term data */ int nQueryTerm; /* Effective size of term in bytes */ int nFullTerm; /* Size of term in bytes incl. tokendata */ Fts5IndexIter *pIter; /* Iterator for this term */ Fts5ExprTerm *pSynonym; /* Pointer to first in list of synonyms */ }; /* ** A phrase. One or more terms that must appear in a contiguous sequence ** within a document for it to match. |
︙ | ︙ | |||
233449 233450 233451 233452 233453 233454 233455 | for(p=pTerm; p; p=p->pSynonym){ int rc; if( p->pIter ){ sqlite3Fts5IterClose(p->pIter); p->pIter = 0; } rc = sqlite3Fts5IndexQuery( | | | 235607 235608 235609 235610 235611 235612 235613 235614 235615 235616 235617 235618 235619 235620 235621 | for(p=pTerm; p; p=p->pSynonym){ int rc; if( p->pIter ){ sqlite3Fts5IterClose(p->pIter); p->pIter = 0; } rc = sqlite3Fts5IndexQuery( pExpr->pIndex, p->pTerm, p->nQueryTerm, (pTerm->bPrefix ? FTS5INDEX_QUERY_PREFIX : 0) | (pExpr->bDesc ? FTS5INDEX_QUERY_DESC : 0), pNear->pColset, &p->pIter ); assert( (rc==SQLITE_OK)==(p->pIter!=0) ); if( rc!=SQLITE_OK ) return rc; |
︙ | ︙ | |||
234086 234087 234088 234089 234090 234091 234092 | static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){ if( pPhrase ){ int i; for(i=0; i<pPhrase->nTerm; i++){ Fts5ExprTerm *pSyn; Fts5ExprTerm *pNext; Fts5ExprTerm *pTerm = &pPhrase->aTerm[i]; | | | 236244 236245 236246 236247 236248 236249 236250 236251 236252 236253 236254 236255 236256 236257 236258 | static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){ if( pPhrase ){ int i; for(i=0; i<pPhrase->nTerm; i++){ Fts5ExprTerm *pSyn; Fts5ExprTerm *pNext; Fts5ExprTerm *pTerm = &pPhrase->aTerm[i]; sqlite3_free(pTerm->pTerm); sqlite3Fts5IterClose(pTerm->pIter); for(pSyn=pTerm->pSynonym; pSyn; pSyn=pNext){ pNext = pSyn->pSynonym; sqlite3Fts5IterClose(pSyn->pIter); fts5BufferFree((Fts5Buffer*)&pSyn[1]); sqlite3_free(pSyn); } |
︙ | ︙ | |||
234184 234185 234186 234187 234188 234189 234190 234191 234192 234193 234194 234195 234196 234197 | } return pRet; } typedef struct TokenCtx TokenCtx; struct TokenCtx { Fts5ExprPhrase *pPhrase; int rc; }; /* ** Callback for tokenizing terms used by ParseTerm(). */ static int fts5ParseTokenize( | > | 236342 236343 236344 236345 236346 236347 236348 236349 236350 236351 236352 236353 236354 236355 236356 | } return pRet; } typedef struct TokenCtx TokenCtx; struct TokenCtx { Fts5ExprPhrase *pPhrase; Fts5Config *pConfig; int rc; }; /* ** Callback for tokenizing terms used by ParseTerm(). */ static int fts5ParseTokenize( |
︙ | ︙ | |||
234217 234218 234219 234220 234221 234222 234223 | Fts5ExprTerm *pSyn; sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1; pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte); if( pSyn==0 ){ rc = SQLITE_NOMEM; }else{ memset(pSyn, 0, (size_t)nByte); | | > > > > | | 236376 236377 236378 236379 236380 236381 236382 236383 236384 236385 236386 236387 236388 236389 236390 236391 236392 236393 236394 236395 | Fts5ExprTerm *pSyn; sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1; pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte); if( pSyn==0 ){ rc = SQLITE_NOMEM; }else{ memset(pSyn, 0, (size_t)nByte); pSyn->pTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer); pSyn->nFullTerm = pSyn->nQueryTerm = nToken; if( pCtx->pConfig->bTokendata ){ pSyn->nQueryTerm = (int)strlen(pSyn->pTerm); } memcpy(pSyn->pTerm, pToken, nToken); pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym; pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn; } }else{ Fts5ExprTerm *pTerm; if( pPhrase==0 || (pPhrase->nTerm % SZALLOC)==0 ){ Fts5ExprPhrase *pNew; |
︙ | ︙ | |||
234243 234244 234245 234246 234247 234248 234249 | pNew->nTerm = nNew - SZALLOC; } } if( rc==SQLITE_OK ){ pTerm = &pPhrase->aTerm[pPhrase->nTerm++]; memset(pTerm, 0, sizeof(Fts5ExprTerm)); | | > > > > | 236406 236407 236408 236409 236410 236411 236412 236413 236414 236415 236416 236417 236418 236419 236420 236421 236422 236423 236424 | pNew->nTerm = nNew - SZALLOC; } } if( rc==SQLITE_OK ){ pTerm = &pPhrase->aTerm[pPhrase->nTerm++]; memset(pTerm, 0, sizeof(Fts5ExprTerm)); pTerm->pTerm = sqlite3Fts5Strndup(&rc, pToken, nToken); pTerm->nFullTerm = pTerm->nQueryTerm = nToken; if( pCtx->pConfig->bTokendata && rc==SQLITE_OK ){ pTerm->nQueryTerm = (int)strlen(pTerm->pTerm); } } } pCtx->rc = rc; return rc; } |
︙ | ︙ | |||
234310 234311 234312 234313 234314 234315 234316 234317 234318 234319 234320 234321 234322 234323 | Fts5Config *pConfig = pParse->pConfig; TokenCtx sCtx; /* Context object passed to callback */ int rc; /* Tokenize return code */ char *z = 0; memset(&sCtx, 0, sizeof(TokenCtx)); sCtx.pPhrase = pAppend; rc = fts5ParseStringFromToken(pToken, &z); if( rc==SQLITE_OK ){ int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_PREFIX : 0); int n; sqlite3Fts5Dequote(z); n = (int)strlen(z); | > | 236477 236478 236479 236480 236481 236482 236483 236484 236485 236486 236487 236488 236489 236490 236491 | Fts5Config *pConfig = pParse->pConfig; TokenCtx sCtx; /* Context object passed to callback */ int rc; /* Tokenize return code */ char *z = 0; memset(&sCtx, 0, sizeof(TokenCtx)); sCtx.pPhrase = pAppend; sCtx.pConfig = pConfig; rc = fts5ParseStringFromToken(pToken, &z); if( rc==SQLITE_OK ){ int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_PREFIX : 0); int n; sqlite3Fts5Dequote(z); n = (int)strlen(z); |
︙ | ︙ | |||
234357 234358 234359 234360 234361 234362 234363 | */ static int sqlite3Fts5ExprClonePhrase( Fts5Expr *pExpr, int iPhrase, Fts5Expr **ppNew ){ int rc = SQLITE_OK; /* Return code */ | | | | > > | | > | > | | > | | | | < | < | | | | | | | | | | | > | 236525 236526 236527 236528 236529 236530 236531 236532 236533 236534 236535 236536 236537 236538 236539 236540 236541 236542 236543 236544 236545 236546 236547 236548 236549 236550 236551 236552 236553 236554 236555 236556 236557 236558 236559 236560 236561 236562 236563 236564 236565 236566 236567 236568 236569 236570 236571 236572 236573 236574 236575 236576 236577 236578 236579 236580 236581 236582 236583 236584 236585 236586 236587 236588 236589 236590 236591 236592 236593 236594 | */ static int sqlite3Fts5ExprClonePhrase( Fts5Expr *pExpr, int iPhrase, Fts5Expr **ppNew ){ int rc = SQLITE_OK; /* Return code */ Fts5ExprPhrase *pOrig = 0; /* The phrase extracted from pExpr */ Fts5Expr *pNew = 0; /* Expression to return via *ppNew */ TokenCtx sCtx = {0,0,0}; /* Context object for fts5ParseTokenize */ if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){ rc = SQLITE_RANGE; }else{ pOrig = pExpr->apExprPhrase[iPhrase]; pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr)); } if( rc==SQLITE_OK ){ pNew->apExprPhrase = (Fts5ExprPhrase**)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase*)); } if( rc==SQLITE_OK ){ pNew->pRoot = (Fts5ExprNode*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprNode)); } if( rc==SQLITE_OK ){ pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprNearset) + sizeof(Fts5ExprPhrase*)); } if( rc==SQLITE_OK && ALWAYS(pOrig!=0) ){ Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset; if( pColsetOrig ){ sqlite3_int64 nByte; Fts5Colset *pColset; nByte = sizeof(Fts5Colset) + (pColsetOrig->nCol-1) * sizeof(int); pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&rc, nByte); if( pColset ){ memcpy(pColset, pColsetOrig, (size_t)nByte); } pNew->pRoot->pNear->pColset = pColset; } } if( rc==SQLITE_OK ){ if( pOrig->nTerm ){ int i; /* Used to iterate through phrase terms */ sCtx.pConfig = pExpr->pConfig; for(i=0; rc==SQLITE_OK && i<pOrig->nTerm; i++){ int tflags = 0; Fts5ExprTerm *p; for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){ rc = fts5ParseTokenize((void*)&sCtx,tflags,p->pTerm,p->nFullTerm,0,0); tflags = FTS5_TOKEN_COLOCATED; } if( rc==SQLITE_OK ){ sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix; sCtx.pPhrase->aTerm[i].bFirst = pOrig->aTerm[i].bFirst; } } }else{ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase)); } } if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){ /* All the allocations succeeded. Put the expression object together. */ pNew->pIndex = pExpr->pIndex; pNew->pConfig = pExpr->pConfig; pNew->nPhrase = 1; |
︙ | ︙ | |||
234778 234779 234780 234781 234782 234783 234784 234785 234786 | Fts5ExprPhrase *pPhrase = (Fts5ExprPhrase*)sqlite3Fts5MallocZero( &pParse->rc, sizeof(Fts5ExprPhrase) ); if( pPhrase ){ if( parseGrowPhraseArray(pParse) ){ fts5ExprPhraseFree(pPhrase); }else{ pParse->apPhrase[pParse->nPhrase++] = pPhrase; pPhrase->nTerm = 1; | > > | < < > > | 236950 236951 236952 236953 236954 236955 236956 236957 236958 236959 236960 236961 236962 236963 236964 236965 236966 236967 236968 236969 236970 | Fts5ExprPhrase *pPhrase = (Fts5ExprPhrase*)sqlite3Fts5MallocZero( &pParse->rc, sizeof(Fts5ExprPhrase) ); if( pPhrase ){ if( parseGrowPhraseArray(pParse) ){ fts5ExprPhraseFree(pPhrase); }else{ Fts5ExprTerm *p = &pNear->apPhrase[0]->aTerm[ii]; Fts5ExprTerm *pTo = &pPhrase->aTerm[0]; pParse->apPhrase[pParse->nPhrase++] = pPhrase; pPhrase->nTerm = 1; pTo->pTerm = sqlite3Fts5Strndup(&pParse->rc, p->pTerm, p->nFullTerm); pTo->nQueryTerm = p->nQueryTerm; pTo->nFullTerm = p->nFullTerm; pRet->apChild[ii] = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 0, 0, sqlite3Fts5ParseNearset(pParse, 0, pPhrase) ); } } } |
︙ | ︙ | |||
234967 234968 234969 234970 234971 234972 234973 | static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){ sqlite3_int64 nByte = 0; Fts5ExprTerm *p; char *zQuoted; /* Determine the maximum amount of space required. */ for(p=pTerm; p; p=p->pSynonym){ | | | > | | 237141 237142 237143 237144 237145 237146 237147 237148 237149 237150 237151 237152 237153 237154 237155 237156 237157 237158 237159 237160 237161 237162 237163 237164 237165 | static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){ sqlite3_int64 nByte = 0; Fts5ExprTerm *p; char *zQuoted; /* Determine the maximum amount of space required. */ for(p=pTerm; p; p=p->pSynonym){ nByte += pTerm->nQueryTerm * 2 + 3 + 2; } zQuoted = sqlite3_malloc64(nByte); if( zQuoted ){ int i = 0; for(p=pTerm; p; p=p->pSynonym){ char *zIn = p->pTerm; char *zEnd = &zIn[p->nQueryTerm]; zQuoted[i++] = '"'; while( zIn<zEnd ){ if( *zIn=='"' ) zQuoted[i++] = '"'; zQuoted[i++] = *zIn++; } zQuoted[i++] = '"'; if( p->pSynonym ) zQuoted[i++] = '|'; } if( pTerm->bPrefix ){ |
︙ | ︙ | |||
235054 235055 235056 235057 235058 235059 235060 | if( zRet==0 ) return 0; for(i=0; i<pNear->nPhrase; i++){ Fts5ExprPhrase *pPhrase = pNear->apPhrase[i]; zRet = fts5PrintfAppend(zRet, " {"); for(iTerm=0; zRet && iTerm<pPhrase->nTerm; iTerm++){ | | | > > | 237229 237230 237231 237232 237233 237234 237235 237236 237237 237238 237239 237240 237241 237242 237243 237244 237245 237246 | if( zRet==0 ) return 0; for(i=0; i<pNear->nPhrase; i++){ Fts5ExprPhrase *pPhrase = pNear->apPhrase[i]; zRet = fts5PrintfAppend(zRet, " {"); for(iTerm=0; zRet && iTerm<pPhrase->nTerm; iTerm++){ Fts5ExprTerm *p = &pPhrase->aTerm[iTerm]; zRet = fts5PrintfAppend(zRet, "%s%.*s", iTerm==0?"":" ", p->nQueryTerm, p->pTerm ); if( pPhrase->aTerm[iTerm].bPrefix ){ zRet = fts5PrintfAppend(zRet, "*"); } } if( zRet ) zRet = fts5PrintfAppend(zRet, "}"); if( zRet==0 ) return 0; |
︙ | ︙ | |||
235455 235456 235457 235458 235459 235460 235461 235462 235463 235464 235465 235466 235467 235468 235469 235470 235471 235472 235473 235474 235475 235476 | static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){ int i; for(i=0; i<pColset->nCol; i++){ if( pColset->aiCol[i]==iCol ) return 1; } return 0; } static int fts5ExprPopulatePoslistsCb( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iUnused1, /* Byte offset of token within input text */ int iUnused2 /* Byte offset of end of token within input text */ ){ Fts5ExprCtx *p = (Fts5ExprCtx*)pCtx; Fts5Expr *pExpr = p->pExpr; int i; UNUSED_PARAM2(iUnused1, iUnused2); | > > > > > > > > > > > > > | > > > | | < | | > > > > > > > | 237632 237633 237634 237635 237636 237637 237638 237639 237640 237641 237642 237643 237644 237645 237646 237647 237648 237649 237650 237651 237652 237653 237654 237655 237656 237657 237658 237659 237660 237661 237662 237663 237664 237665 237666 237667 237668 237669 237670 237671 237672 237673 237674 237675 237676 237677 237678 237679 237680 237681 237682 237683 237684 237685 237686 237687 237688 237689 237690 237691 237692 237693 237694 237695 | static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){ int i; for(i=0; i<pColset->nCol; i++){ if( pColset->aiCol[i]==iCol ) return 1; } return 0; } /* ** pToken is a buffer nToken bytes in size that may or may not contain ** an embedded 0x00 byte. If it does, return the number of bytes in ** the buffer before the 0x00. If it does not, return nToken. */ static int fts5QueryTerm(const char *pToken, int nToken){ int ii; for(ii=0; ii<nToken && pToken[ii]; ii++){} return ii; } static int fts5ExprPopulatePoslistsCb( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iUnused1, /* Byte offset of token within input text */ int iUnused2 /* Byte offset of end of token within input text */ ){ Fts5ExprCtx *p = (Fts5ExprCtx*)pCtx; Fts5Expr *pExpr = p->pExpr; int i; int nQuery = nToken; i64 iRowid = pExpr->pRoot->iRowid; UNUSED_PARAM2(iUnused1, iUnused2); if( nQuery>FTS5_MAX_TOKEN_SIZE ) nQuery = FTS5_MAX_TOKEN_SIZE; if( pExpr->pConfig->bTokendata ){ nQuery = fts5QueryTerm(pToken, nQuery); } if( (tflags & FTS5_TOKEN_COLOCATED)==0 ) p->iOff++; for(i=0; i<pExpr->nPhrase; i++){ Fts5ExprTerm *pT; if( p->aPopulator[i].bOk==0 ) continue; for(pT=&pExpr->apExprPhrase[i]->aTerm[0]; pT; pT=pT->pSynonym){ if( (pT->nQueryTerm==nQuery || (pT->nQueryTerm<nQuery && pT->bPrefix)) && memcmp(pT->pTerm, pToken, pT->nQueryTerm)==0 ){ int rc = sqlite3Fts5PoslistWriterAppend( &pExpr->apExprPhrase[i]->poslist, &p->aPopulator[i].writer, p->iOff ); if( rc==SQLITE_OK && pExpr->pConfig->bTokendata && !pT->bPrefix ){ int iCol = p->iOff>>32; int iTokOff = p->iOff & 0x7FFFFFFF; rc = sqlite3Fts5IndexIterWriteTokendata( pT->pIter, pToken, nToken, iRowid, iCol, iTokOff ); } if( rc ) return rc; break; } } } return SQLITE_OK; } |
︙ | ︙ | |||
235617 235618 235619 235620 235621 235622 235623 235624 235625 235626 235627 235628 235629 235630 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } /* ** 2014 August 11 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 237816 237817 237818 237819 237820 237821 237822 237823 237824 237825 237826 237827 237828 237829 237830 237831 237832 237833 237834 237835 237836 237837 237838 237839 237840 237841 237842 237843 237844 237845 237846 237847 237848 237849 237850 237851 237852 237853 237854 237855 237856 237857 237858 237859 237860 237861 237862 237863 237864 237865 237866 237867 237868 237869 237870 237871 237872 237873 237874 237875 237876 237877 237878 237879 237880 237881 237882 237883 237884 237885 237886 237887 237888 237889 237890 237891 237892 237893 237894 237895 237896 237897 237898 237899 237900 237901 237902 237903 237904 237905 237906 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } /* ** Does the work of the fts5_api.xQueryToken() API method. */ static int sqlite3Fts5ExprQueryToken( Fts5Expr *pExpr, int iPhrase, int iToken, const char **ppOut, int *pnOut ){ Fts5ExprPhrase *pPhrase = 0; if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){ return SQLITE_RANGE; } pPhrase = pExpr->apExprPhrase[iPhrase]; if( iToken<0 || iToken>=pPhrase->nTerm ){ return SQLITE_RANGE; } *ppOut = pPhrase->aTerm[iToken].pTerm; *pnOut = pPhrase->aTerm[iToken].nFullTerm; return SQLITE_OK; } /* ** Does the work of the fts5_api.xInstToken() API method. */ static int sqlite3Fts5ExprInstToken( Fts5Expr *pExpr, i64 iRowid, int iPhrase, int iCol, int iOff, int iToken, const char **ppOut, int *pnOut ){ Fts5ExprPhrase *pPhrase = 0; Fts5ExprTerm *pTerm = 0; int rc = SQLITE_OK; if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){ return SQLITE_RANGE; } pPhrase = pExpr->apExprPhrase[iPhrase]; if( iToken<0 || iToken>=pPhrase->nTerm ){ return SQLITE_RANGE; } pTerm = &pPhrase->aTerm[iToken]; if( pTerm->bPrefix==0 ){ if( pExpr->pConfig->bTokendata ){ rc = sqlite3Fts5IterToken( pTerm->pIter, iRowid, iCol, iOff+iToken, ppOut, pnOut ); }else{ *ppOut = pTerm->pTerm; *pnOut = pTerm->nFullTerm; } } return rc; } /* ** Clear the token mappings for all Fts5IndexIter objects mannaged by ** the expression passed as the only argument. */ static void sqlite3Fts5ExprClearTokens(Fts5Expr *pExpr){ int ii; for(ii=0; ii<pExpr->nPhrase; ii++){ Fts5ExprTerm *pT; for(pT=&pExpr->apExprPhrase[ii]->aTerm[0]; pT; pT=pT->pSynonym){ sqlite3Fts5IndexIterClearTokendata(pT->pIter); } } } /* ** 2014 August 11 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** |
︙ | ︙ | |||
235656 235657 235658 235659 235660 235661 235662 | int nSlot; /* Size of aSlot[] array */ Fts5HashEntry *pScan; /* Current ordered scan item */ Fts5HashEntry **aSlot; /* Array of hash slots */ }; /* ** Each entry in the hash table is represented by an object of the | | | | | > > > > > | 237932 237933 237934 237935 237936 237937 237938 237939 237940 237941 237942 237943 237944 237945 237946 237947 237948 237949 237950 237951 237952 237953 237954 | int nSlot; /* Size of aSlot[] array */ Fts5HashEntry *pScan; /* Current ordered scan item */ Fts5HashEntry **aSlot; /* Array of hash slots */ }; /* ** Each entry in the hash table is represented by an object of the ** following type. Each object, its key, and its current data are stored ** in a single memory allocation. The key immediately follows the object ** in memory. The position list data immediately follows the key data ** in memory. ** ** The key is Fts5HashEntry.nKey bytes in size. It consists of a single ** byte identifying the index (either the main term index or a prefix-index), ** followed by the term data. For example: "0token". There is no ** nul-terminator - in this case nKey=6. ** ** The data that follows the key is in a similar, but not identical format ** to the doclist data stored in the database. It is: ** ** * Rowid, as a varint ** * Position list, without 0x00 terminator. ** * Size of previous position list and rowid, as a 4 byte |
︙ | ︙ | |||
235794 235795 235796 235797 235798 235799 235800 | memset(apNew, 0, nNew*sizeof(Fts5HashEntry*)); for(i=0; i<pHash->nSlot; i++){ while( apOld[i] ){ unsigned int iHash; Fts5HashEntry *p = apOld[i]; apOld[i] = p->pHashNext; | | < | 238075 238076 238077 238078 238079 238080 238081 238082 238083 238084 238085 238086 238087 238088 238089 | memset(apNew, 0, nNew*sizeof(Fts5HashEntry*)); for(i=0; i<pHash->nSlot; i++){ while( apOld[i] ){ unsigned int iHash; Fts5HashEntry *p = apOld[i]; apOld[i] = p->pHashNext; iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p), p->nKey); p->pHashNext = apNew[iHash]; apNew[iHash] = p; } } sqlite3_free(apOld); pHash->nSlot = nNew; |
︙ | ︙ | |||
235879 235880 235881 235882 235883 235884 235885 | bNew = (pHash->eDetail==FTS5_DETAIL_FULL); /* Attempt to locate an existing hash entry */ iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken); for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ char *zKey = fts5EntryKey(p); if( zKey[0]==bByte | | | 238159 238160 238161 238162 238163 238164 238165 238166 238167 238168 238169 238170 238171 238172 238173 | bNew = (pHash->eDetail==FTS5_DETAIL_FULL); /* Attempt to locate an existing hash entry */ iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken); for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ char *zKey = fts5EntryKey(p); if( zKey[0]==bByte && p->nKey==nToken+1 && memcmp(&zKey[1], pToken, nToken)==0 ){ break; } } /* If an existing hash entry cannot be found, create a new one. */ |
︙ | ︙ | |||
235909 235910 235911 235912 235913 235914 235915 | if( !p ) return SQLITE_NOMEM; memset(p, 0, sizeof(Fts5HashEntry)); p->nAlloc = (int)nByte; zKey = fts5EntryKey(p); zKey[0] = bByte; memcpy(&zKey[1], pToken, nToken); assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) ); | | | | 238189 238190 238191 238192 238193 238194 238195 238196 238197 238198 238199 238200 238201 238202 238203 238204 238205 | if( !p ) return SQLITE_NOMEM; memset(p, 0, sizeof(Fts5HashEntry)); p->nAlloc = (int)nByte; zKey = fts5EntryKey(p); zKey[0] = bByte; memcpy(&zKey[1], pToken, nToken); assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) ); p->nKey = nToken+1; zKey[nToken+1] = '\0'; p->nData = nToken+1 + sizeof(Fts5HashEntry); p->pHashNext = pHash->aSlot[iHash]; pHash->aSlot[iHash] = p; pHash->nEntry++; /* Add the first rowid field to the hash-entry */ p->nData += sqlite3Fts5PutVarint(&((u8*)p)[p->nData], iRowid); p->iRowid = iRowid; |
︙ | ︙ | |||
236028 236029 236030 236031 236032 236033 236034 | if( p1==0 ){ *ppOut = p2; p2 = 0; }else if( p2==0 ){ *ppOut = p1; p1 = 0; }else{ | < > | > > > | > | > | | < < | 238308 238309 238310 238311 238312 238313 238314 238315 238316 238317 238318 238319 238320 238321 238322 238323 238324 238325 238326 238327 238328 238329 238330 238331 238332 238333 238334 238335 238336 238337 238338 238339 238340 238341 238342 238343 238344 238345 238346 238347 238348 238349 238350 238351 238352 | if( p1==0 ){ *ppOut = p2; p2 = 0; }else if( p2==0 ){ *ppOut = p1; p1 = 0; }else{ char *zKey1 = fts5EntryKey(p1); char *zKey2 = fts5EntryKey(p2); int nMin = MIN(p1->nKey, p2->nKey); int cmp = memcmp(zKey1, zKey2, nMin); if( cmp==0 ){ cmp = p1->nKey - p2->nKey; } assert( cmp!=0 ); if( cmp>0 ){ /* p2 is smaller */ *ppOut = p2; ppOut = &p2->pScanNext; p2 = p2->pScanNext; }else{ /* p1 is smaller */ *ppOut = p1; ppOut = &p1->pScanNext; p1 = p1->pScanNext; } *ppOut = 0; } } return pRet; } /* ** Link all tokens from hash table iHash into a list in sorted order. The ** tokens are not removed from the hash table. */ static int fts5HashEntrySort( Fts5Hash *pHash, const char *pTerm, int nTerm, /* Query prefix, if any */ Fts5HashEntry **ppSorted ){ const int nMergeSlot = 32; |
︙ | ︙ | |||
236077 236078 236079 236080 236081 236082 236083 | if( !ap ) return SQLITE_NOMEM; memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot); for(iSlot=0; iSlot<pHash->nSlot; iSlot++){ Fts5HashEntry *pIter; for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){ if( pTerm==0 | | | 238360 238361 238362 238363 238364 238365 238366 238367 238368 238369 238370 238371 238372 238373 238374 | if( !ap ) return SQLITE_NOMEM; memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot); for(iSlot=0; iSlot<pHash->nSlot; iSlot++){ Fts5HashEntry *pIter; for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){ if( pTerm==0 || (pIter->nKey>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm)) ){ Fts5HashEntry *pEntry = pIter; pEntry->pScanNext = 0; for(i=0; ap[i]; i++){ pEntry = fts5HashEntryMerge(pEntry, ap[i]); ap[i] = 0; } |
︙ | ︙ | |||
236116 236117 236118 236119 236120 236121 236122 | ){ unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm); char *zKey = 0; Fts5HashEntry *p; for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ zKey = fts5EntryKey(p); | < | | | 238399 238400 238401 238402 238403 238404 238405 238406 238407 238408 238409 238410 238411 238412 238413 238414 238415 238416 238417 | ){ unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm); char *zKey = 0; Fts5HashEntry *p; for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ zKey = fts5EntryKey(p); if( nTerm==p->nKey && memcmp(zKey, pTerm, nTerm)==0 ) break; } if( p ){ int nHashPre = sizeof(Fts5HashEntry) + nTerm; int nList = p->nData - nHashPre; u8 *pRet = (u8*)(*ppOut = sqlite3_malloc64(nPre + nList + 10)); if( pRet ){ Fts5HashEntry *pFaux = (Fts5HashEntry*)&pRet[nPre-nHashPre]; memcpy(&pRet[nPre], &((u8*)p)[nHashPre], nList); nList += fts5HashAddPoslistSize(pHash, p, pFaux); *pnDoclist = nList; |
︙ | ︙ | |||
236182 236183 236184 236185 236186 236187 236188 236189 236190 236191 236192 236193 236194 | static int sqlite3Fts5HashScanEof(Fts5Hash *p){ return (p->pScan==0); } static void sqlite3Fts5HashScanEntry( Fts5Hash *pHash, const char **pzTerm, /* OUT: term (nul-terminated) */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); | > | > | | > | 238464 238465 238466 238467 238468 238469 238470 238471 238472 238473 238474 238475 238476 238477 238478 238479 238480 238481 238482 238483 238484 238485 238486 238487 238488 238489 238490 238491 238492 238493 | static int sqlite3Fts5HashScanEof(Fts5Hash *p){ return (p->pScan==0); } static void sqlite3Fts5HashScanEntry( Fts5Hash *pHash, const char **pzTerm, /* OUT: term (nul-terminated) */ int *pnTerm, /* OUT: Size of term in bytes */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); int nTerm = p->nKey; fts5HashAddPoslistSize(pHash, p, 0); *pzTerm = zKey; *pnTerm = nTerm; *ppDoclist = (const u8*)&zKey[nTerm]; *pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm); }else{ *pzTerm = 0; *pnTerm = 0; *ppDoclist = 0; *pnDoclist = 0; } } /* ** 2014 May 31 |
︙ | ︙ | |||
236525 236526 236527 236528 236529 236530 236531 236532 236533 236534 236535 236536 236537 236538 | typedef struct Fts5PageWriter Fts5PageWriter; typedef struct Fts5SegIter Fts5SegIter; typedef struct Fts5DoclistIter Fts5DoclistIter; typedef struct Fts5SegWriter Fts5SegWriter; typedef struct Fts5Structure Fts5Structure; typedef struct Fts5StructureLevel Fts5StructureLevel; typedef struct Fts5StructureSegment Fts5StructureSegment; struct Fts5Data { u8 *p; /* Pointer to buffer containing record */ int nn; /* Size of record in bytes */ int szLeaf; /* Size of leaf without page-index */ }; | > > > | 238810 238811 238812 238813 238814 238815 238816 238817 238818 238819 238820 238821 238822 238823 238824 238825 238826 | typedef struct Fts5PageWriter Fts5PageWriter; typedef struct Fts5SegIter Fts5SegIter; typedef struct Fts5DoclistIter Fts5DoclistIter; typedef struct Fts5SegWriter Fts5SegWriter; typedef struct Fts5Structure Fts5Structure; typedef struct Fts5StructureLevel Fts5StructureLevel; typedef struct Fts5StructureSegment Fts5StructureSegment; typedef struct Fts5TokenDataIter Fts5TokenDataIter; typedef struct Fts5TokenDataMap Fts5TokenDataMap; typedef struct Fts5TombstoneArray Fts5TombstoneArray; struct Fts5Data { u8 *p; /* Pointer to buffer containing record */ int nn; /* Size of record in bytes */ int szLeaf; /* Size of leaf without page-index */ }; |
︙ | ︙ | |||
236559 236560 236561 236562 236563 236564 236565 236566 236567 236568 236569 236570 236571 236572 236573 236574 236575 236576 236577 236578 236579 236580 | i64 iWriteRowid; /* Rowid for current doc being written */ int bDelete; /* Current write is a delete */ int nContentlessDelete; /* Number of contentless delete ops */ int nPendingRow; /* Number of INSERT in hash table */ /* Error state. */ int rc; /* Current error code */ /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ sqlite3_stmt *pIdxDeleter; /* "DELETE FROM %_idx WHERE segid=?" */ sqlite3_stmt *pIdxSelect; int nRead; /* Total number of blocks read */ sqlite3_stmt *pDeleteFromIdx; sqlite3_stmt *pDataVersion; i64 iStructVersion; /* data_version when pStruct read */ Fts5Structure *pStruct; /* Current db structure (or NULL) */ | > > | 238847 238848 238849 238850 238851 238852 238853 238854 238855 238856 238857 238858 238859 238860 238861 238862 238863 238864 238865 238866 238867 238868 238869 238870 | i64 iWriteRowid; /* Rowid for current doc being written */ int bDelete; /* Current write is a delete */ int nContentlessDelete; /* Number of contentless delete ops */ int nPendingRow; /* Number of INSERT in hash table */ /* Error state. */ int rc; /* Current error code */ int flushRc; /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ sqlite3_stmt *pIdxDeleter; /* "DELETE FROM %_idx WHERE segid=?" */ sqlite3_stmt *pIdxSelect; sqlite3_stmt *pIdxNextSelect; int nRead; /* Total number of blocks read */ sqlite3_stmt *pDeleteFromIdx; sqlite3_stmt *pDataVersion; i64 iStructVersion; /* data_version when pStruct read */ Fts5Structure *pStruct; /* Current db structure (or NULL) */ |
︙ | ︙ | |||
236720 236721 236722 236723 236724 236725 236726 | struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ i64 iLeafOffset; /* Byte offset within current leaf */ | | < | 239010 239011 239012 239013 239014 239015 239016 239017 239018 239019 239020 239021 239022 239023 239024 | struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ i64 iLeafOffset; /* Byte offset within current leaf */ Fts5TombstoneArray *pTombArray; /* Array of tombstone pages */ /* Next method */ void (*xNext)(Fts5Index*, Fts5SegIter*, int*); /* The page and offset from which the current term was read. The offset ** is the offset of the first rowid in the current doclist. */ int iTermLeafPgno; |
︙ | ︙ | |||
236747 236748 236749 236750 236751 236752 236753 236754 236755 236756 236757 236758 236759 236760 | /* Variables populated based on current entry. */ Fts5Buffer term; /* Current term */ i64 iRowid; /* Current rowid */ int nPos; /* Number of bytes in current position list */ u8 bDel; /* True if the delete flag is set */ }; /* ** Argument is a pointer to an Fts5Data structure that contains a ** leaf page. */ #define ASSERT_SZLEAF_OK(x) assert( \ (x)->szLeaf==(x)->nn || (x)->szLeaf==fts5GetU16(&(x)->p[2]) \ | > > > > > > > > > | 239036 239037 239038 239039 239040 239041 239042 239043 239044 239045 239046 239047 239048 239049 239050 239051 239052 239053 239054 239055 239056 239057 239058 | /* Variables populated based on current entry. */ Fts5Buffer term; /* Current term */ i64 iRowid; /* Current rowid */ int nPos; /* Number of bytes in current position list */ u8 bDel; /* True if the delete flag is set */ }; /* ** Array of tombstone pages. Reference counted. */ struct Fts5TombstoneArray { int nRef; /* Number of pointers to this object */ int nTombstone; Fts5Data *apTombstone[1]; /* Array of tombstone pages */ }; /* ** Argument is a pointer to an Fts5Data structure that contains a ** leaf page. */ #define ASSERT_SZLEAF_OK(x) assert( \ (x)->szLeaf==(x)->nn || (x)->szLeaf==fts5GetU16(&(x)->p[2]) \ |
︙ | ︙ | |||
236792 236793 236794 236795 236796 236797 236798 236799 236800 236801 236802 236803 236804 236805 236806 236807 236808 236809 236810 236811 236812 236813 236814 236815 236816 236817 | ** ** aFirst[1] contains the index in aSeg[] of the iterator that points to ** the smallest key overall. aFirst[0] is unused. ** ** poslist: ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ Fts5Colset *pColset; /* Restrict matches to these columns */ /* Invoked to set output variables. */ void (*xSetOutputs)(Fts5Iter*, Fts5SegIter*); int nSeg; /* Size of aSeg[] array */ int bRev; /* True to iterate in reverse order */ u8 bSkipEmpty; /* True to skip deleted entries */ i64 iSwitchRowid; /* Firstest rowid of other than aFirst[1] */ Fts5CResult *aFirst; /* Current merge state (see above) */ Fts5SegIter aSeg[1]; /* Array of segment iterators */ }; | > > > > > > > < | 239090 239091 239092 239093 239094 239095 239096 239097 239098 239099 239100 239101 239102 239103 239104 239105 239106 239107 239108 239109 239110 239111 239112 239113 239114 239115 239116 239117 239118 239119 239120 239121 239122 239123 239124 239125 239126 239127 239128 239129 | ** ** aFirst[1] contains the index in aSeg[] of the iterator that points to ** the smallest key overall. aFirst[0] is unused. ** ** poslist: ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. ** ** pColset: ** If not NULL, points to an object containing a set of column indices. ** Only matches that occur in one of these columns will be returned. ** The Fts5Iter does not own the Fts5Colset object, and so it is not ** freed when the iterator is closed - it is owned by the upper layer. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5TokenDataIter *pTokenDataIter; Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ Fts5Colset *pColset; /* Restrict matches to these columns */ /* Invoked to set output variables. */ void (*xSetOutputs)(Fts5Iter*, Fts5SegIter*); int nSeg; /* Size of aSeg[] array */ int bRev; /* True to iterate in reverse order */ u8 bSkipEmpty; /* True to skip deleted entries */ i64 iSwitchRowid; /* Firstest rowid of other than aFirst[1] */ Fts5CResult *aFirst; /* Current merge state (see above) */ Fts5SegIter aSeg[1]; /* Array of segment iterators */ }; /* ** An instance of the following type is used to iterate through the contents ** of a doclist-index record. ** ** pData: ** Record containing the doclist-index data. |
︙ | ︙ | |||
237730 237731 237732 237733 237734 237735 237736 | }else{ int iOff; for(iOff=pLvl->iOff; iOff<pData->nn; iOff++){ if( pData->p[iOff] ) break; } if( iOff<pData->nn ){ | | | | 240034 240035 240036 240037 240038 240039 240040 240041 240042 240043 240044 240045 240046 240047 240048 240049 240050 | }else{ int iOff; for(iOff=pLvl->iOff; iOff<pData->nn; iOff++){ if( pData->p[iOff] ) break; } if( iOff<pData->nn ){ u64 iVal; pLvl->iLeafPgno += (iOff - pLvl->iOff) + 1; iOff += fts5GetVarint(&pData->p[iOff], &iVal); pLvl->iRowid += iVal; pLvl->iOff = iOff; }else{ pLvl->bEof = 1; } } |
︙ | ︙ | |||
238111 238112 238113 238114 238115 238116 238117 | pIter->xNext = fts5SegIterNext_None; }else{ pIter->xNext = fts5SegIterNext; } } /* | | | | | > | | < | > > | 240415 240416 240417 240418 240419 240420 240421 240422 240423 240424 240425 240426 240427 240428 240429 240430 240431 240432 240433 240434 240435 240436 240437 240438 240439 240440 240441 240442 | pIter->xNext = fts5SegIterNext_None; }else{ pIter->xNext = fts5SegIterNext; } } /* ** Allocate a tombstone hash page array object (pIter->pTombArray) for ** the iterator passed as the second argument. If an OOM error occurs, ** leave an error in the Fts5Index object. */ static void fts5SegIterAllocTombstone(Fts5Index *p, Fts5SegIter *pIter){ const int nTomb = pIter->pSeg->nPgTombstone; if( nTomb>0 ){ int nByte = nTomb * sizeof(Fts5Data*) + sizeof(Fts5TombstoneArray); Fts5TombstoneArray *pNew; pNew = (Fts5TombstoneArray*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pNew ){ pNew->nTombstone = nTomb; pNew->nRef = 1; pIter->pTombArray = pNew; } } } /* ** Initialize the iterator object pIter to iterate through the entries in ** segment pSeg. The iterator is left pointing to the first entry when |
︙ | ︙ | |||
238379 238380 238381 238382 238383 238384 238385 238386 238387 | iOff += fts5GetVarint32(&pIter->pLeaf->p[iOff], nKeep); } pIter->iLeafOffset = iOff; fts5SegIterLoadTerm(p, pIter, nKeep); }else{ const u8 *pList = 0; const char *zTerm = 0; int nList; sqlite3Fts5HashScanNext(p->pHash); | > | | | 240685 240686 240687 240688 240689 240690 240691 240692 240693 240694 240695 240696 240697 240698 240699 240700 240701 240702 240703 240704 240705 240706 240707 240708 | iOff += fts5GetVarint32(&pIter->pLeaf->p[iOff], nKeep); } pIter->iLeafOffset = iOff; fts5SegIterLoadTerm(p, pIter, nKeep); }else{ const u8 *pList = 0; const char *zTerm = 0; int nTerm = 0; int nList; sqlite3Fts5HashScanNext(p->pHash); sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList); if( pList==0 ) goto next_none_eof; pIter->pLeaf->p = (u8*)pList; pIter->pLeaf->nn = nList; pIter->pLeaf->szLeaf = nList; pIter->iEndofDoclist = nList; sqlite3Fts5BufferSet(&p->rc,&pIter->term, nTerm, (u8*)zTerm); pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid); } if( pbNewTerm ) *pbNewTerm = 1; }else{ goto next_none_eof; } |
︙ | ︙ | |||
238453 238454 238455 238456 238457 238458 238459 238460 238461 238462 238463 | assert_nc( iDelta>0 ); } pIter->iLeafOffset = iOff; }else if( pIter->pSeg==0 ){ const u8 *pList = 0; const char *zTerm = 0; int nList = 0; assert( (pIter->flags & FTS5_SEGITER_ONETERM) || pbNewTerm ); if( 0==(pIter->flags & FTS5_SEGITER_ONETERM) ){ sqlite3Fts5HashScanNext(p->pHash); | > | | < | 240760 240761 240762 240763 240764 240765 240766 240767 240768 240769 240770 240771 240772 240773 240774 240775 240776 240777 240778 240779 240780 240781 240782 240783 240784 240785 240786 240787 240788 240789 | assert_nc( iDelta>0 ); } pIter->iLeafOffset = iOff; }else if( pIter->pSeg==0 ){ const u8 *pList = 0; const char *zTerm = 0; int nTerm = 0; int nList = 0; assert( (pIter->flags & FTS5_SEGITER_ONETERM) || pbNewTerm ); if( 0==(pIter->flags & FTS5_SEGITER_ONETERM) ){ sqlite3Fts5HashScanNext(p->pHash); sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList); } if( pList==0 ){ fts5DataRelease(pIter->pLeaf); pIter->pLeaf = 0; }else{ pIter->pLeaf->p = (u8*)pList; pIter->pLeaf->nn = nList; pIter->pLeaf->szLeaf = nList; pIter->iEndofDoclist = nList+1; sqlite3Fts5BufferSet(&p->rc, &pIter->term, nTerm, (u8*)zTerm); pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid); *pbNewTerm = 1; } }else{ iOff = 0; /* Next entry is not on the current page */ while( iOff==0 ){ |
︙ | ︙ | |||
238854 238855 238856 238857 238858 238859 238860 | pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf ){ fts5LeafSeek(p, bGe, pIter, pTerm, nTerm); } | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 241161 241162 241163 241164 241165 241166 241167 241168 241169 241170 241171 241172 241173 241174 241175 241176 241177 241178 241179 241180 241181 241182 241183 241184 241185 241186 241187 241188 241189 241190 241191 241192 241193 241194 241195 241196 241197 241198 241199 241200 241201 241202 241203 241204 241205 241206 241207 241208 241209 241210 241211 241212 241213 241214 241215 241216 241217 241218 241219 241220 241221 241222 241223 241224 241225 241226 241227 241228 241229 241230 241231 241232 241233 241234 241235 241236 241237 241238 241239 241240 241241 241242 241243 241244 241245 241246 241247 241248 241249 241250 241251 241252 241253 241254 241255 241256 241257 241258 241259 241260 241261 241262 241263 241264 241265 241266 241267 241268 241269 241270 241271 241272 241273 241274 241275 241276 241277 241278 241279 241280 241281 241282 | pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf ){ fts5LeafSeek(p, bGe, pIter, pTerm, nTerm); } if( p->rc==SQLITE_OK && (bGe==0 || (flags & FTS5INDEX_QUERY_SCANONETERM)) ){ pIter->flags |= FTS5_SEGITER_ONETERM; if( pIter->pLeaf ){ if( flags & FTS5INDEX_QUERY_DESC ){ pIter->flags |= FTS5_SEGITER_REVERSE; } if( bDlidx ){ fts5SegIterLoadDlidx(p, pIter); } if( flags & FTS5INDEX_QUERY_DESC ){ fts5SegIterReverse(p, pIter); } } } fts5SegIterSetNext(p, pIter); if( 0==(flags & FTS5INDEX_QUERY_SCANONETERM) ){ fts5SegIterAllocTombstone(p, pIter); } /* Either: ** ** 1) an error has occurred, or ** 2) the iterator points to EOF, or ** 3) the iterator points to an entry with term (pTerm/nTerm), or ** 4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points ** to an entry with a term greater than or equal to (pTerm/nTerm). */ assert_nc( p->rc!=SQLITE_OK /* 1 */ || pIter->pLeaf==0 /* 2 */ || fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)==0 /* 3 */ || (bGe && fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)>0) /* 4 */ ); } /* ** SQL used by fts5SegIterNextInit() to find the page to open. */ static sqlite3_stmt *fts5IdxNextStmt(Fts5Index *p){ if( p->pIdxNextSelect==0 ){ Fts5Config *pConfig = p->pConfig; fts5IndexPrepareStmt(p, &p->pIdxNextSelect, sqlite3_mprintf( "SELECT pgno FROM '%q'.'%q_idx' WHERE " "segid=? AND term>? ORDER BY term ASC LIMIT 1", pConfig->zDb, pConfig->zName )); } return p->pIdxNextSelect; } /* ** This is similar to fts5SegIterSeekInit(), except that it initializes ** the segment iterator to point to the first term following the page ** with pToken/nToken on it. */ static void fts5SegIterNextInit( Fts5Index *p, const char *pTerm, int nTerm, Fts5StructureSegment *pSeg, /* Description of segment */ Fts5SegIter *pIter /* Object to populate */ ){ int iPg = -1; /* Page of segment to open */ int bDlidx = 0; sqlite3_stmt *pSel = 0; /* SELECT to find iPg */ pSel = fts5IdxNextStmt(p); if( pSel ){ assert( p->rc==SQLITE_OK ); sqlite3_bind_int(pSel, 1, pSeg->iSegid); sqlite3_bind_blob(pSel, 2, pTerm, nTerm, SQLITE_STATIC); if( sqlite3_step(pSel)==SQLITE_ROW ){ i64 val = sqlite3_column_int64(pSel, 0); iPg = (int)(val>>1); bDlidx = (val & 0x0001); } p->rc = sqlite3_reset(pSel); sqlite3_bind_null(pSel, 2); if( p->rc ) return; } memset(pIter, 0, sizeof(*pIter)); pIter->pSeg = pSeg; pIter->flags |= FTS5_SEGITER_ONETERM; if( iPg>=0 ){ pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); fts5SegIterSetNext(p, pIter); } if( pIter->pLeaf ){ const u8 *a = pIter->pLeaf->p; int iTermOff = 0; pIter->iPgidxOff = pIter->pLeaf->szLeaf; pIter->iPgidxOff += fts5GetVarint32(&a[pIter->iPgidxOff], iTermOff); pIter->iLeafOffset = iTermOff; fts5SegIterLoadTerm(p, pIter, 0); fts5SegIterLoadNPos(p, pIter); if( bDlidx ) fts5SegIterLoadDlidx(p, pIter); assert( p->rc!=SQLITE_OK || fts5BufferCompareBlob(&pIter->term, (const u8*)pTerm, nTerm)>0 ); } } /* ** Initialize the object pIter to point to term pTerm/nTerm within the ** in-memory hash table. If there is no such term in the hash-table, the ** iterator is set to EOF. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If ** an error has already occurred when this function is called, it is a no-op. |
︙ | ︙ | |||
238913 238914 238915 238916 238917 238918 238919 | assert( p->pHash ); assert( p->rc==SQLITE_OK ); if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){ const u8 *pList = 0; p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm); | | < > > > > > > > > | 241295 241296 241297 241298 241299 241300 241301 241302 241303 241304 241305 241306 241307 241308 241309 241310 241311 241312 241313 241314 241315 241316 241317 241318 241319 241320 241321 241322 241323 | assert( p->pHash ); assert( p->rc==SQLITE_OK ); if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){ const u8 *pList = 0; p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm); sqlite3Fts5HashScanEntry(p->pHash, (const char**)&z, &n, &pList, &nList); if( pList ){ pLeaf = fts5IdxMalloc(p, sizeof(Fts5Data)); if( pLeaf ){ pLeaf->p = (u8*)pList; } } /* The call to sqlite3Fts5HashScanInit() causes the hash table to ** fill the size field of all existing position lists. This means they ** can no longer be appended to. Since the only scenario in which they ** can be appended to is if the previous operation on this table was ** a DELETE, by clearing the Fts5Index.bDelete flag we can avoid this ** possibility altogether. */ p->bDelete = 0; }else{ p->rc = sqlite3Fts5HashQuery(p->pHash, sizeof(Fts5Data), (const char*)pTerm, nTerm, (void**)&pLeaf, &nList ); if( pLeaf ){ pLeaf->p = (u8*)&pLeaf[1]; } |
︙ | ︙ | |||
238964 238965 238966 238967 238968 238969 238970 238971 238972 238973 238974 238975 238976 238977 238978 | int ii; for(ii=0; ii<n; ii++){ fts5DataRelease(ap[ii]); } sqlite3_free(ap); } } /* ** Zero the iterator passed as the only argument. */ static void fts5SegIterClear(Fts5SegIter *pIter){ fts5BufferFree(&pIter->term); fts5DataRelease(pIter->pLeaf); fts5DataRelease(pIter->pNextLeaf); | > > > > > > > > > > > > > > > > > | | 241353 241354 241355 241356 241357 241358 241359 241360 241361 241362 241363 241364 241365 241366 241367 241368 241369 241370 241371 241372 241373 241374 241375 241376 241377 241378 241379 241380 241381 241382 241383 241384 241385 241386 241387 241388 241389 241390 241391 241392 | int ii; for(ii=0; ii<n; ii++){ fts5DataRelease(ap[ii]); } sqlite3_free(ap); } } /* ** Decrement the ref-count of the object passed as the only argument. If it ** reaches 0, free it and its contents. */ static void fts5TombstoneArrayDelete(Fts5TombstoneArray *p){ if( p ){ p->nRef--; if( p->nRef<=0 ){ int ii; for(ii=0; ii<p->nTombstone; ii++){ fts5DataRelease(p->apTombstone[ii]); } sqlite3_free(p); } } } /* ** Zero the iterator passed as the only argument. */ static void fts5SegIterClear(Fts5SegIter *pIter){ fts5BufferFree(&pIter->term); fts5DataRelease(pIter->pLeaf); fts5DataRelease(pIter->pNextLeaf); fts5TombstoneArrayDelete(pIter->pTombArray); fts5DlidxIterFree(pIter->pDlidx); sqlite3_free(pIter->aRowidOffset); memset(pIter, 0, sizeof(Fts5SegIter)); } #ifdef SQLITE_DEBUG |
︙ | ︙ | |||
239217 239218 239219 239220 239221 239222 239223 | if( pIter->pLeaf==0 ) break; if( bRev==0 && pIter->iRowid>=iMatch ) break; if( bRev!=0 && pIter->iRowid<=iMatch ) break; bMove = 1; }while( p->rc==SQLITE_OK ); } | < | 241623 241624 241625 241626 241627 241628 241629 241630 241631 241632 241633 241634 241635 241636 | if( pIter->pLeaf==0 ) break; if( bRev==0 && pIter->iRowid>=iMatch ) break; if( bRev!=0 && pIter->iRowid<=iMatch ) break; bMove = 1; }while( p->rc==SQLITE_OK ); } /* ** Free the iterator object passed as the second argument. */ static void fts5MultiIterFree(Fts5Iter *pIter){ if( pIter ){ int i; for(i=0; i<pIter->nSeg; i++){ |
︙ | ︙ | |||
239362 239363 239364 239365 239366 239367 239368 239369 | ** Return true if the iterator passed as the only argument points ** to an segment entry for which there is a tombstone. Return false ** if there is no tombstone or if the iterator is already at EOF. */ static int fts5MultiIterIsDeleted(Fts5Iter *pIter){ int iFirst = pIter->aFirst[1].iFirst; Fts5SegIter *pSeg = &pIter->aSeg[iFirst]; | > | | | | | | | | 241767 241768 241769 241770 241771 241772 241773 241774 241775 241776 241777 241778 241779 241780 241781 241782 241783 241784 241785 241786 241787 241788 241789 241790 241791 241792 241793 241794 241795 241796 241797 241798 241799 | ** Return true if the iterator passed as the only argument points ** to an segment entry for which there is a tombstone. Return false ** if there is no tombstone or if the iterator is already at EOF. */ static int fts5MultiIterIsDeleted(Fts5Iter *pIter){ int iFirst = pIter->aFirst[1].iFirst; Fts5SegIter *pSeg = &pIter->aSeg[iFirst]; Fts5TombstoneArray *pArray = pSeg->pTombArray; if( pSeg->pLeaf && pArray ){ /* Figure out which page the rowid might be present on. */ int iPg = ((u64)pSeg->iRowid) % pArray->nTombstone; assert( iPg>=0 ); /* If tombstone hash page iPg has not yet been loaded from the ** database, load it now. */ if( pArray->apTombstone[iPg]==0 ){ pArray->apTombstone[iPg] = fts5DataRead(pIter->pIndex, FTS5_TOMBSTONE_ROWID(pSeg->pSeg->iSegid, iPg) ); if( pArray->apTombstone[iPg]==0 ) return 0; } return fts5IndexTombstoneQuery( pArray->apTombstone[iPg], pArray->nTombstone, pSeg->iRowid ); } return 0; } |
︙ | ︙ | |||
239918 239919 239920 239921 239922 239923 239924 239925 239926 239927 239928 239929 239930 239931 | }else{ pIter->xSetOutputs = fts5IterSetOutputs_Col; } } } } /* ** Allocate a new Fts5Iter object. ** ** The new object will be used to iterate through data in structure pStruct. ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel ** is zero or greater, data from the first nSegment segments on level iLevel | > > > > > > > > > > > > > > > > > > > > > > > > > > | 242324 242325 242326 242327 242328 242329 242330 242331 242332 242333 242334 242335 242336 242337 242338 242339 242340 242341 242342 242343 242344 242345 242346 242347 242348 242349 242350 242351 242352 242353 242354 242355 242356 242357 242358 242359 242360 242361 242362 242363 | }else{ pIter->xSetOutputs = fts5IterSetOutputs_Col; } } } } /* ** All the component segment-iterators of pIter have been set up. This ** functions finishes setup for iterator pIter itself. */ static void fts5MultiIterFinishSetup(Fts5Index *p, Fts5Iter *pIter){ int iIter; for(iIter=pIter->nSeg-1; iIter>0; iIter--){ int iEq; if( (iEq = fts5MultiIterDoCompare(pIter, iIter)) ){ Fts5SegIter *pSeg = &pIter->aSeg[iEq]; if( p->rc==SQLITE_OK ) pSeg->xNext(p, pSeg, 0); fts5MultiIterAdvanced(p, pIter, iEq, iIter); } } fts5MultiIterSetEof(pIter); fts5AssertMultiIterSetup(p, pIter); if( (pIter->bSkipEmpty && fts5MultiIterIsEmpty(p, pIter)) || fts5MultiIterIsDeleted(pIter) ){ fts5MultiIterNext(p, pIter, 0, 0); }else if( pIter->base.bEof==0 ){ Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; pIter->xSetOutputs(pIter, pSeg); } } /* ** Allocate a new Fts5Iter object. ** ** The new object will be used to iterate through data in structure pStruct. ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel ** is zero or greater, data from the first nSegment segments on level iLevel |
︙ | ︙ | |||
239999 240000 240001 240002 240003 240004 240005 | for(iSeg=nSeg-1; iSeg>=0; iSeg--){ fts5SegIterInit(p, &pLvl->aSeg[iSeg], &pNew->aSeg[iIter++]); } } assert( iIter==nSeg ); } | | < < < < < < < < | < < < < < < < < < < < | 242431 242432 242433 242434 242435 242436 242437 242438 242439 242440 242441 242442 242443 242444 242445 242446 242447 242448 242449 242450 | for(iSeg=nSeg-1; iSeg>=0; iSeg--){ fts5SegIterInit(p, &pLvl->aSeg[iSeg], &pNew->aSeg[iIter++]); } } assert( iIter==nSeg ); } /* If the above was successful, each component iterator now points ** to the first entry in its segment. In this case initialize the ** aFirst[] array. Or, if an error has occurred, free the iterator ** object and set the output variable to NULL. */ if( p->rc==SQLITE_OK ){ fts5MultiIterFinishSetup(p, pNew); }else{ fts5MultiIterFree(pNew); *ppOut = 0; } fts5MultiIterNew_post_check: assert( (*ppOut)!=0 || p->rc!=SQLITE_OK ); |
︙ | ︙ | |||
240048 240049 240050 240051 240052 240053 240054 | int bDesc, /* True for descending rowid order */ Fts5Iter **ppOut /* New object */ ){ Fts5Iter *pNew; pNew = fts5MultiIterAlloc(p, 2); if( pNew ){ Fts5SegIter *pIter = &pNew->aSeg[1]; | < | 242461 242462 242463 242464 242465 242466 242467 242468 242469 242470 242471 242472 242473 242474 | int bDesc, /* True for descending rowid order */ Fts5Iter **ppOut /* New object */ ){ Fts5Iter *pNew; pNew = fts5MultiIterAlloc(p, 2); if( pNew ){ Fts5SegIter *pIter = &pNew->aSeg[1]; pIter->flags = FTS5_SEGITER_ONETERM; if( pData->szLeaf>0 ){ pIter->pLeaf = pData; pIter->iLeafOffset = fts5GetVarint(pData->p, (u64*)&pIter->iRowid); pIter->iEndofDoclist = pData->nn; pNew->aFirst[1].iFirst = 1; if( bDesc ){ |
︙ | ︙ | |||
240196 240197 240198 240199 240200 240201 240202 240203 240204 240205 240206 240207 240208 240209 | */ static void fts5IndexDiscardData(Fts5Index *p){ assert( p->pHash || p->nPendingData==0 ); if( p->pHash ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; } p->nContentlessDelete = 0; } /* ** Return the size of the prefix, in bytes, that buffer ** (pNew/<length-unknown>) shares with buffer (pOld/nOld). | > | 242608 242609 242610 242611 242612 242613 242614 242615 242616 242617 242618 242619 242620 242621 242622 | */ static void fts5IndexDiscardData(Fts5Index *p){ assert( p->pHash || p->nPendingData==0 ); if( p->pHash ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; p->flushRc = SQLITE_OK; } p->nContentlessDelete = 0; } /* ** Return the size of the prefix, in bytes, that buffer ** (pNew/<length-unknown>) shares with buffer (pOld/nOld). |
︙ | ︙ | |||
240411 240412 240413 240414 240415 240416 240417 | pDlidx->bPrevValid = 0; pDlidx->pgno++; }else{ bDone = 1; } if( pDlidx->bPrevValid ){ | | | 242824 242825 242826 242827 242828 242829 242830 242831 242832 242833 242834 242835 242836 242837 242838 | pDlidx->bPrevValid = 0; pDlidx->pgno++; }else{ bDone = 1; } if( pDlidx->bPrevValid ){ iVal = (u64)iRowid - (u64)pDlidx->iPrev; }else{ i64 iPgno = (i==0 ? pWriter->writer.pgno : pDlidx[-1].pgno); assert( pDlidx->buf.n==0 ); sqlite3Fts5BufferAppendVarint(&p->rc, &pDlidx->buf, !bDone); sqlite3Fts5BufferAppendVarint(&p->rc, &pDlidx->buf, iPgno); iVal = iRowid; } |
︙ | ︙ | |||
240598 240599 240600 240601 240602 240603 240604 | const u8 *aData, int nData ){ Fts5PageWriter *pPage = &pWriter->writer; const u8 *a = aData; int n = nData; | | | 243011 243012 243013 243014 243015 243016 243017 243018 243019 243020 243021 243022 243023 243024 243025 | const u8 *aData, int nData ){ Fts5PageWriter *pPage = &pWriter->writer; const u8 *a = aData; int n = nData; assert( p->pConfig->pgsz>0 || p->rc!=SQLITE_OK ); while( p->rc==SQLITE_OK && (pPage->buf.n + pPage->pgidx.n + n)>=p->pConfig->pgsz ){ int nReq = p->pConfig->pgsz - pPage->buf.n - pPage->pgidx.n; int nCopy = 0; while( nCopy<nReq ){ i64 dummy; |
︙ | ︙ | |||
241331 241332 241333 241334 241335 241336 241337 | assert_nc( iSOP==pSeg->iLeafOffset ); iNextOff = pSeg->iLeafOffset + pSeg->nPos; } } iOff = iStart; | > > > > | | < | | | | > | > > | | 243744 243745 243746 243747 243748 243749 243750 243751 243752 243753 243754 243755 243756 243757 243758 243759 243760 243761 243762 243763 243764 243765 243766 243767 243768 243769 243770 243771 243772 243773 243774 243775 | assert_nc( iSOP==pSeg->iLeafOffset ); iNextOff = pSeg->iLeafOffset + pSeg->nPos; } } iOff = iStart; /* If the position-list for the entry being removed flows over past ** the end of this page, delete the portion of the position-list on the ** next page and beyond. ** ** Set variable bLastInDoclist to true if this entry happens ** to be the last rowid in the doclist for its term. */ if( iNextOff>=iPgIdx ){ int pgno = pSeg->iLeafPgno+1; fts5SecureDeleteOverflow(p, pSeg->pSeg, pgno, &bLastInDoclist); iNextOff = iPgIdx; } if( pSeg->bDel==0 ){ if( iNextOff!=iPgIdx ){ /* Loop through the page-footer. If iNextOff (offset of the ** entry following the one we are removing) is equal to the ** offset of a key on this page, then the entry is the last ** in its doclist. */ int iKeyOff = 0; for(iIdx=0; iIdx<nIdx; /* no-op */){ u32 iVal = 0; iIdx += fts5GetVarint32(&aIdx[iIdx], iVal); iKeyOff += iVal; if( iKeyOff==iNextOff ){ bLastInDoclist = 1; |
︙ | ︙ | |||
241528 241529 241530 241531 241532 241533 241534 241535 241536 241537 | ** mode. It edits the segments within the database described by argument ** pStruct to remove the entries for term zTerm, rowid iRowid. */ static void fts5FlushSecureDelete( Fts5Index *p, Fts5Structure *pStruct, const char *zTerm, i64 iRowid ){ const int f = FTS5INDEX_QUERY_SKIPHASH; | > < | 243947 243948 243949 243950 243951 243952 243953 243954 243955 243956 243957 243958 243959 243960 243961 243962 243963 243964 | ** mode. It edits the segments within the database described by argument ** pStruct to remove the entries for term zTerm, rowid iRowid. */ static void fts5FlushSecureDelete( Fts5Index *p, Fts5Structure *pStruct, const char *zTerm, int nTerm, i64 iRowid ){ const int f = FTS5INDEX_QUERY_SKIPHASH; Fts5Iter *pIter = 0; /* Used to find term instance */ fts5MultiIterNew(p, pStruct, f, 0, (const u8*)zTerm, nTerm, -1, 0, &pIter); if( fts5MultiIterEof(p, pIter)==0 ){ i64 iThis = fts5MultiIterRowid(pIter); if( iThis<iRowid ){ fts5MultiIterNextFrom(p, pIter, iRowid); |
︙ | ︙ | |||
241605 241606 241607 241608 241609 241610 241611 | while( p->rc==SQLITE_OK && 0==sqlite3Fts5HashScanEof(pHash) ){ const char *zTerm; /* Buffer containing term */ int nTerm; /* Size of zTerm in bytes */ const u8 *pDoclist; /* Pointer to doclist for this term */ int nDoclist; /* Size of doclist in bytes */ /* Get the term and doclist for this entry. */ | | < | 244024 244025 244026 244027 244028 244029 244030 244031 244032 244033 244034 244035 244036 244037 244038 | while( p->rc==SQLITE_OK && 0==sqlite3Fts5HashScanEof(pHash) ){ const char *zTerm; /* Buffer containing term */ int nTerm; /* Size of zTerm in bytes */ const u8 *pDoclist; /* Pointer to doclist for this term */ int nDoclist; /* Size of doclist in bytes */ /* Get the term and doclist for this entry. */ sqlite3Fts5HashScanEntry(pHash, &zTerm, &nTerm, &pDoclist, &nDoclist); if( bSecureDelete==0 ){ fts5WriteAppendTerm(p, &writer, nTerm, (const u8*)zTerm); if( p->rc!=SQLITE_OK ) break; assert( writer.bFirstRowidInPage==0 ); } if( !bSecureDelete && pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){ |
︙ | ︙ | |||
241636 241637 241638 241639 241640 241641 241642 | /* If in secure delete mode, and if this entry in the poslist is ** in fact a delete, then edit the existing segments directly ** using fts5FlushSecureDelete(). */ if( bSecureDelete ){ if( eDetail==FTS5_DETAIL_NONE ){ if( iOff<nDoclist && pDoclist[iOff]==0x00 ){ | | | | 244054 244055 244056 244057 244058 244059 244060 244061 244062 244063 244064 244065 244066 244067 244068 244069 244070 244071 244072 244073 244074 244075 244076 244077 244078 | /* If in secure delete mode, and if this entry in the poslist is ** in fact a delete, then edit the existing segments directly ** using fts5FlushSecureDelete(). */ if( bSecureDelete ){ if( eDetail==FTS5_DETAIL_NONE ){ if( iOff<nDoclist && pDoclist[iOff]==0x00 ){ fts5FlushSecureDelete(p, pStruct, zTerm, nTerm, iRowid); iOff++; if( iOff<nDoclist && pDoclist[iOff]==0x00 ){ iOff++; nDoclist = 0; }else{ continue; } } }else if( (pDoclist[iOff] & 0x01) ){ fts5FlushSecureDelete(p, pStruct, zTerm, nTerm, iRowid); if( p->rc!=SQLITE_OK || pDoclist[iOff]==0x01 ){ iOff++; continue; } } } |
︙ | ︙ | |||
241772 241773 241774 241775 241776 241777 241778 241779 241780 241781 241782 241783 241784 241785 241786 241787 241788 241789 241790 241791 241792 241793 | } /* ** Flush any data stored in the in-memory hash tables to the database. */ static void fts5IndexFlush(Fts5Index *p){ /* Unless it is empty, flush the hash table to disk */ if( p->nPendingData || p->nContentlessDelete ){ assert( p->pHash ); fts5FlushOneHash(p); if( p->rc==SQLITE_OK ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; p->nContentlessDelete = 0; } } } static Fts5Structure *fts5IndexOptimizeStruct( Fts5Index *p, Fts5Structure *pStruct | > > > > > > | 244190 244191 244192 244193 244194 244195 244196 244197 244198 244199 244200 244201 244202 244203 244204 244205 244206 244207 244208 244209 244210 244211 244212 244213 244214 244215 244216 244217 | } /* ** Flush any data stored in the in-memory hash tables to the database. */ static void fts5IndexFlush(Fts5Index *p){ /* Unless it is empty, flush the hash table to disk */ if( p->flushRc ){ p->rc = p->flushRc; return; } if( p->nPendingData || p->nContentlessDelete ){ assert( p->pHash ); fts5FlushOneHash(p); if( p->rc==SQLITE_OK ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; p->nContentlessDelete = 0; }else if( p->nPendingData || p->nContentlessDelete ){ p->flushRc = p->rc; } } } static Fts5Structure *fts5IndexOptimizeStruct( Fts5Index *p, Fts5Structure *pStruct |
︙ | ︙ | |||
241858 241859 241860 241861 241862 241863 241864 | static int sqlite3Fts5IndexOptimize(Fts5Index *p){ Fts5Structure *pStruct; Fts5Structure *pNew = 0; assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); | | > | 244282 244283 244284 244285 244286 244287 244288 244289 244290 244291 244292 244293 244294 244295 244296 244297 244298 | static int sqlite3Fts5IndexOptimize(Fts5Index *p){ Fts5Structure *pStruct; Fts5Structure *pNew = 0; assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); assert( p->rc!=SQLITE_OK || p->nContentlessDelete==0 ); pStruct = fts5StructureRead(p); assert( p->rc!=SQLITE_OK || pStruct!=0 ); fts5StructureInvalidate(p); if( pStruct ){ pNew = fts5IndexOptimizeStruct(p, pStruct); } fts5StructureRelease(pStruct); |
︙ | ︙ | |||
242265 242266 242267 242268 242269 242270 242271 | static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ int iIdx, /* Index to scan for data */ u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset, /* Restrict matches to these columns */ | | > | > > > > > > | 244690 244691 244692 244693 244694 244695 244696 244697 244698 244699 244700 244701 244702 244703 244704 244705 244706 244707 244708 244709 244710 244711 244712 244713 244714 244715 244716 244717 244718 244719 244720 244721 244722 244723 244724 244725 244726 244727 244728 244729 244730 244731 244732 244733 244734 244735 244736 244737 244738 244739 244740 244741 244742 244743 244744 | static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ int iIdx, /* Index to scan for data */ u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset, /* Restrict matches to these columns */ Fts5Iter **ppIter /* OUT: New iterator */ ){ Fts5Structure *pStruct; Fts5Buffer *aBuf; int nBuf = 32; int nMerge = 1; void (*xMerge)(Fts5Index*, Fts5Buffer*, int, Fts5Buffer*); void (*xAppend)(Fts5Index*, u64, Fts5Iter*, Fts5Buffer*); if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){ xMerge = fts5MergeRowidLists; xAppend = fts5AppendRowid; }else{ nMerge = FTS5_MERGE_NLIST-1; nBuf = nMerge*8; /* Sufficient to merge (16^8)==(2^32) lists */ xMerge = fts5MergePrefixLists; xAppend = fts5AppendPoslist; } aBuf = (Fts5Buffer*)fts5IdxMalloc(p, sizeof(Fts5Buffer)*nBuf); pStruct = fts5StructureRead(p); assert( p->rc!=SQLITE_OK || (aBuf && pStruct) ); if( p->rc==SQLITE_OK ){ const int flags = FTS5INDEX_QUERY_SCAN | FTS5INDEX_QUERY_SKIPEMPTY | FTS5INDEX_QUERY_NOOUTPUT; int i; i64 iLastRowid = 0; Fts5Iter *p1 = 0; /* Iterator used to gather data from index */ Fts5Data *pData; Fts5Buffer doclist; int bNewTerm = 1; memset(&doclist, 0, sizeof(doclist)); /* If iIdx is non-zero, then it is the number of a prefix-index for ** prefixes 1 character longer than the prefix being queried for. That ** index contains all the doclists required, except for the one ** corresponding to the prefix itself. That one is extracted from the ** main term index here. */ if( iIdx!=0 ){ int dummy = 0; const int f2 = FTS5INDEX_QUERY_SKIPEMPTY|FTS5INDEX_QUERY_NOOUTPUT; pToken[0] = FTS5_MAIN_PREFIX; fts5MultiIterNew(p, pStruct, f2, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for(; |
︙ | ︙ | |||
242322 242323 242324 242325 242326 242327 242328 242329 242330 242331 242332 242333 242334 242335 242336 242337 242338 242339 242340 242341 242342 242343 | } fts5MultiIterFree(p1); } pToken[0] = FTS5_MAIN_PREFIX + iIdx; fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for( /* no-op */ ; fts5MultiIterEof(p, p1)==0; fts5MultiIterNext2(p, p1, &bNewTerm) ){ Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ]; int nTerm = pSeg->term.n; const u8 *pTerm = pSeg->term.p; p1->xSetOutputs(p1, pSeg); assert_nc( memcmp(pToken, pTerm, MIN(nToken, nTerm))<=0 ); if( bNewTerm ){ if( nTerm<nToken || memcmp(pToken, pTerm, nToken) ) break; } if( p1->base.nData==0 ) continue; | > < | 244754 244755 244756 244757 244758 244759 244760 244761 244762 244763 244764 244765 244766 244767 244768 244769 244770 244771 244772 244773 244774 244775 244776 244777 244778 244779 244780 244781 244782 244783 | } fts5MultiIterFree(p1); } pToken[0] = FTS5_MAIN_PREFIX + iIdx; fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for( /* no-op */ ; fts5MultiIterEof(p, p1)==0; fts5MultiIterNext2(p, p1, &bNewTerm) ){ Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ]; int nTerm = pSeg->term.n; const u8 *pTerm = pSeg->term.p; p1->xSetOutputs(p1, pSeg); assert_nc( memcmp(pToken, pTerm, MIN(nToken, nTerm))<=0 ); if( bNewTerm ){ if( nTerm<nToken || memcmp(pToken, pTerm, nToken) ) break; } if( p1->base.nData==0 ) continue; if( p1->base.iRowid<=iLastRowid && doclist.n>0 ){ for(i=0; p->rc==SQLITE_OK && doclist.n; i++){ int i1 = i*nMerge; int iStore; assert( i1+nMerge<=nBuf ); for(iStore=i1; iStore<i1+nMerge; iStore++){ if( aBuf[iStore].n==0 ){ |
︙ | ︙ | |||
242376 242377 242378 242379 242380 242381 242382 | } for(iFree=i; iFree<i+nMerge; iFree++){ fts5BufferFree(&aBuf[iFree]); } } fts5MultiIterFree(p1); | | | 244808 244809 244810 244811 244812 244813 244814 244815 244816 244817 244818 244819 244820 244821 244822 | } for(iFree=i; iFree<i+nMerge; iFree++){ fts5BufferFree(&aBuf[iFree]); } } fts5MultiIterFree(p1); pData = fts5IdxMalloc(p, sizeof(*pData)+doclist.n+FTS5_DATA_ZERO_PADDING); if( pData ){ pData->p = (u8*)&pData[1]; pData->nn = pData->szLeaf = doclist.n; if( doclist.n ) memcpy(pData->p, doclist.p, doclist.n); fts5MultiIterNew2(p, pData, bDesc, ppIter); } fts5BufferFree(&doclist); |
︙ | ︙ | |||
242519 242520 242521 242522 242523 242524 242525 242526 242527 242528 242529 242530 242531 242532 | assert( p->pReader==0 ); fts5StructureInvalidate(p); sqlite3_finalize(p->pWriter); sqlite3_finalize(p->pDeleter); sqlite3_finalize(p->pIdxWriter); sqlite3_finalize(p->pIdxDeleter); sqlite3_finalize(p->pIdxSelect); sqlite3_finalize(p->pDataVersion); sqlite3_finalize(p->pDeleteFromIdx); sqlite3Fts5HashFree(p->pHash); sqlite3_free(p->zDataTbl); sqlite3_free(p); } return rc; | > | 244951 244952 244953 244954 244955 244956 244957 244958 244959 244960 244961 244962 244963 244964 244965 | assert( p->pReader==0 ); fts5StructureInvalidate(p); sqlite3_finalize(p->pWriter); sqlite3_finalize(p->pDeleter); sqlite3_finalize(p->pIdxWriter); sqlite3_finalize(p->pIdxDeleter); sqlite3_finalize(p->pIdxSelect); sqlite3_finalize(p->pIdxNextSelect); sqlite3_finalize(p->pDataVersion); sqlite3_finalize(p->pDeleteFromIdx); sqlite3Fts5HashFree(p->pHash); sqlite3_free(p->zDataTbl); sqlite3_free(p); } return rc; |
︙ | ︙ | |||
242613 242614 242615 242616 242617 242618 242619 242620 242621 242622 242623 242624 242625 242626 | nByte ); } } return rc; } /* ** Open a new iterator to iterate though all rowid that match the ** specified token or token prefix. */ static int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 245046 245047 245048 245049 245050 245051 245052 245053 245054 245055 245056 245057 245058 245059 245060 245061 245062 245063 245064 245065 245066 245067 245068 245069 245070 245071 245072 245073 245074 245075 245076 245077 245078 245079 245080 245081 245082 245083 245084 245085 245086 245087 245088 245089 245090 245091 245092 245093 245094 245095 245096 245097 245098 245099 245100 245101 245102 245103 245104 245105 245106 245107 245108 245109 245110 245111 245112 245113 245114 245115 245116 245117 245118 245119 245120 245121 245122 245123 245124 245125 245126 245127 245128 245129 245130 245131 245132 245133 245134 245135 245136 245137 245138 245139 245140 245141 245142 245143 245144 245145 245146 245147 245148 245149 245150 245151 245152 245153 245154 245155 245156 245157 245158 245159 245160 245161 245162 245163 245164 245165 245166 245167 245168 245169 245170 245171 245172 245173 245174 245175 245176 245177 245178 245179 245180 245181 245182 245183 245184 245185 245186 245187 245188 245189 245190 245191 245192 245193 245194 245195 245196 245197 245198 245199 245200 245201 245202 245203 245204 245205 245206 245207 245208 245209 245210 245211 245212 245213 245214 245215 245216 245217 245218 245219 245220 245221 245222 245223 245224 245225 245226 245227 245228 245229 245230 245231 245232 245233 245234 245235 245236 245237 245238 245239 245240 245241 245242 245243 245244 245245 245246 245247 245248 245249 245250 245251 245252 245253 245254 245255 245256 245257 245258 245259 245260 245261 245262 245263 245264 245265 245266 245267 245268 245269 245270 245271 245272 245273 245274 245275 245276 245277 245278 245279 245280 245281 245282 245283 245284 245285 245286 245287 245288 245289 245290 245291 245292 245293 245294 245295 245296 245297 245298 245299 245300 245301 245302 245303 245304 245305 245306 245307 245308 245309 245310 245311 245312 245313 245314 245315 245316 245317 245318 245319 245320 245321 245322 245323 245324 245325 245326 245327 245328 245329 245330 245331 245332 245333 245334 245335 245336 245337 245338 245339 245340 245341 245342 245343 245344 245345 245346 245347 245348 245349 245350 245351 245352 245353 245354 245355 245356 245357 245358 245359 245360 245361 245362 245363 245364 245365 245366 245367 245368 245369 245370 245371 245372 245373 245374 245375 245376 245377 245378 245379 245380 245381 245382 245383 245384 245385 245386 245387 245388 245389 245390 245391 245392 245393 245394 245395 245396 245397 245398 245399 245400 245401 245402 245403 245404 245405 245406 245407 245408 245409 245410 245411 245412 245413 245414 245415 245416 245417 245418 245419 245420 245421 245422 245423 245424 245425 245426 245427 245428 245429 245430 245431 245432 245433 245434 245435 245436 245437 245438 245439 245440 245441 245442 245443 245444 245445 245446 245447 245448 245449 245450 245451 245452 245453 245454 245455 245456 245457 245458 245459 245460 245461 245462 245463 245464 245465 245466 245467 245468 245469 245470 245471 245472 245473 245474 245475 245476 245477 245478 245479 245480 245481 245482 245483 245484 245485 245486 245487 245488 245489 245490 245491 245492 245493 245494 245495 245496 245497 245498 245499 245500 245501 245502 245503 245504 245505 245506 245507 | nByte ); } } return rc; } /* ** pToken points to a buffer of size nToken bytes containing a search ** term, including the index number at the start, used on a tokendata=1 ** table. This function returns true if the term in buffer pBuf matches ** token pToken/nToken. */ static int fts5IsTokendataPrefix( Fts5Buffer *pBuf, const u8 *pToken, int nToken ){ return ( pBuf->n>=nToken && 0==memcmp(pBuf->p, pToken, nToken) && (pBuf->n==nToken || pBuf->p[nToken]==0x00) ); } /* ** Ensure the segment-iterator passed as the only argument points to EOF. */ static void fts5SegIterSetEOF(Fts5SegIter *pSeg){ fts5DataRelease(pSeg->pLeaf); pSeg->pLeaf = 0; } /* ** Usually, a tokendata=1 iterator (struct Fts5TokenDataIter) accumulates an ** array of these for each row it visits. Or, for an iterator used by an ** "ORDER BY rank" query, it accumulates an array of these for the entire ** query. ** ** Each instance in the array indicates the iterator (and therefore term) ** associated with position iPos of rowid iRowid. This is used by the ** xInstToken() API. */ struct Fts5TokenDataMap { i64 iRowid; /* Row this token is located in */ i64 iPos; /* Position of token */ int iIter; /* Iterator token was read from */ }; /* ** An object used to supplement Fts5Iter for tokendata=1 iterators. */ struct Fts5TokenDataIter { int nIter; int nIterAlloc; int nMap; int nMapAlloc; Fts5TokenDataMap *aMap; Fts5PoslistReader *aPoslistReader; int *aPoslistToIter; Fts5Iter *apIter[1]; }; /* ** This function appends iterator pAppend to Fts5TokenDataIter pIn and ** returns the result. */ static Fts5TokenDataIter *fts5AppendTokendataIter( Fts5Index *p, /* Index object (for error code) */ Fts5TokenDataIter *pIn, /* Current Fts5TokenDataIter struct */ Fts5Iter *pAppend /* Append this iterator */ ){ Fts5TokenDataIter *pRet = pIn; if( p->rc==SQLITE_OK ){ if( pIn==0 || pIn->nIter==pIn->nIterAlloc ){ int nAlloc = pIn ? pIn->nIterAlloc*2 : 16; int nByte = nAlloc * sizeof(Fts5Iter*) + sizeof(Fts5TokenDataIter); Fts5TokenDataIter *pNew = (Fts5TokenDataIter*)sqlite3_realloc(pIn, nByte); if( pNew==0 ){ p->rc = SQLITE_NOMEM; }else{ if( pIn==0 ) memset(pNew, 0, nByte); pRet = pNew; pNew->nIterAlloc = nAlloc; } } } if( p->rc ){ sqlite3Fts5IterClose((Fts5IndexIter*)pAppend); }else{ pRet->apIter[pRet->nIter++] = pAppend; } assert( pRet==0 || pRet->nIter<=pRet->nIterAlloc ); return pRet; } /* ** Delete an Fts5TokenDataIter structure and its contents. */ static void fts5TokendataIterDelete(Fts5TokenDataIter *pSet){ if( pSet ){ int ii; for(ii=0; ii<pSet->nIter; ii++){ fts5MultiIterFree(pSet->apIter[ii]); } sqlite3_free(pSet->aPoslistReader); sqlite3_free(pSet->aMap); sqlite3_free(pSet); } } /* ** Append a mapping to the token-map belonging to object pT. */ static void fts5TokendataIterAppendMap( Fts5Index *p, Fts5TokenDataIter *pT, int iIter, i64 iRowid, i64 iPos ){ if( p->rc==SQLITE_OK ){ if( pT->nMap==pT->nMapAlloc ){ int nNew = pT->nMapAlloc ? pT->nMapAlloc*2 : 64; int nByte = nNew * sizeof(Fts5TokenDataMap); Fts5TokenDataMap *aNew; aNew = (Fts5TokenDataMap*)sqlite3_realloc(pT->aMap, nByte); if( aNew==0 ){ p->rc = SQLITE_NOMEM; return; } pT->aMap = aNew; pT->nMapAlloc = nNew; } pT->aMap[pT->nMap].iRowid = iRowid; pT->aMap[pT->nMap].iPos = iPos; pT->aMap[pT->nMap].iIter = iIter; pT->nMap++; } } /* ** The iterator passed as the only argument must be a tokendata=1 iterator ** (pIter->pTokenDataIter!=0). This function sets the iterator output ** variables (pIter->base.*) according to the contents of the current ** row. */ static void fts5IterSetOutputsTokendata(Fts5Iter *pIter){ int ii; int nHit = 0; i64 iRowid = SMALLEST_INT64; int iMin = 0; Fts5TokenDataIter *pT = pIter->pTokenDataIter; pIter->base.nData = 0; pIter->base.pData = 0; for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( p->base.bEof==0 ){ if( nHit==0 || p->base.iRowid<iRowid ){ iRowid = p->base.iRowid; nHit = 1; pIter->base.pData = p->base.pData; pIter->base.nData = p->base.nData; iMin = ii; }else if( p->base.iRowid==iRowid ){ nHit++; } } } if( nHit==0 ){ pIter->base.bEof = 1; }else{ int eDetail = pIter->pIndex->pConfig->eDetail; pIter->base.bEof = 0; pIter->base.iRowid = iRowid; if( nHit==1 && eDetail==FTS5_DETAIL_FULL ){ fts5TokendataIterAppendMap(pIter->pIndex, pT, iMin, iRowid, -1); }else if( nHit>1 && eDetail!=FTS5_DETAIL_NONE ){ int nReader = 0; int nByte = 0; i64 iPrev = 0; /* Allocate array of iterators if they are not already allocated. */ if( pT->aPoslistReader==0 ){ pT->aPoslistReader = (Fts5PoslistReader*)sqlite3Fts5MallocZero( &pIter->pIndex->rc, pT->nIter * (sizeof(Fts5PoslistReader) + sizeof(int)) ); if( pT->aPoslistReader==0 ) return; pT->aPoslistToIter = (int*)&pT->aPoslistReader[pT->nIter]; } /* Populate an iterator for each poslist that will be merged */ for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( iRowid==p->base.iRowid ){ pT->aPoslistToIter[nReader] = ii; sqlite3Fts5PoslistReaderInit( p->base.pData, p->base.nData, &pT->aPoslistReader[nReader++] ); nByte += p->base.nData; } } /* Ensure the output buffer is large enough */ if( fts5BufferGrow(&pIter->pIndex->rc, &pIter->poslist, nByte+nHit*10) ){ return; } /* Ensure the token-mapping is large enough */ if( eDetail==FTS5_DETAIL_FULL && pT->nMapAlloc<(pT->nMap + nByte) ){ int nNew = (pT->nMapAlloc + nByte) * 2; Fts5TokenDataMap *aNew = (Fts5TokenDataMap*)sqlite3_realloc( pT->aMap, nNew*sizeof(Fts5TokenDataMap) ); if( aNew==0 ){ pIter->pIndex->rc = SQLITE_NOMEM; return; } pT->aMap = aNew; pT->nMapAlloc = nNew; } pIter->poslist.n = 0; while( 1 ){ i64 iMinPos = LARGEST_INT64; /* Find smallest position */ iMin = 0; for(ii=0; ii<nReader; ii++){ Fts5PoslistReader *pReader = &pT->aPoslistReader[ii]; if( pReader->bEof==0 ){ if( pReader->iPos<iMinPos ){ iMinPos = pReader->iPos; iMin = ii; } } } /* If all readers were at EOF, break out of the loop. */ if( iMinPos==LARGEST_INT64 ) break; sqlite3Fts5PoslistSafeAppend(&pIter->poslist, &iPrev, iMinPos); sqlite3Fts5PoslistReaderNext(&pT->aPoslistReader[iMin]); if( eDetail==FTS5_DETAIL_FULL ){ pT->aMap[pT->nMap].iPos = iMinPos; pT->aMap[pT->nMap].iIter = pT->aPoslistToIter[iMin]; pT->aMap[pT->nMap].iRowid = iRowid; pT->nMap++; } } pIter->base.pData = pIter->poslist.p; pIter->base.nData = pIter->poslist.n; } } } /* ** The iterator passed as the only argument must be a tokendata=1 iterator ** (pIter->pTokenDataIter!=0). This function advances the iterator. If ** argument bFrom is false, then the iterator is advanced to the next ** entry. Or, if bFrom is true, it is advanced to the first entry with ** a rowid of iFrom or greater. */ static void fts5TokendataIterNext(Fts5Iter *pIter, int bFrom, i64 iFrom){ int ii; Fts5TokenDataIter *pT = pIter->pTokenDataIter; for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( p->base.bEof==0 && (p->base.iRowid==pIter->base.iRowid || (bFrom && p->base.iRowid<iFrom)) ){ fts5MultiIterNext(p->pIndex, p, bFrom, iFrom); while( bFrom && p->base.bEof==0 && p->base.iRowid<iFrom && p->pIndex->rc==SQLITE_OK ){ fts5MultiIterNext(p->pIndex, p, 0, 0); } } } fts5IterSetOutputsTokendata(pIter); } /* ** If the segment-iterator passed as the first argument is at EOF, then ** set pIter->term to a copy of buffer pTerm. */ static void fts5TokendataSetTermIfEof(Fts5Iter *pIter, Fts5Buffer *pTerm){ if( pIter && pIter->aSeg[0].pLeaf==0 ){ fts5BufferSet(&pIter->pIndex->rc, &pIter->aSeg[0].term, pTerm->n, pTerm->p); } } /* ** This function sets up an iterator to use for a non-prefix query on a ** tokendata=1 table. */ static Fts5Iter *fts5SetupTokendataIter( Fts5Index *p, /* FTS index to query */ const u8 *pToken, /* Buffer containing query term */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset /* Colset to filter on */ ){ Fts5Iter *pRet = 0; Fts5TokenDataIter *pSet = 0; Fts5Structure *pStruct = 0; const int flags = FTS5INDEX_QUERY_SCANONETERM | FTS5INDEX_QUERY_SCAN; Fts5Buffer bSeek = {0, 0, 0}; Fts5Buffer *pSmall = 0; fts5IndexFlush(p); pStruct = fts5StructureRead(p); while( p->rc==SQLITE_OK ){ Fts5Iter *pPrev = pSet ? pSet->apIter[pSet->nIter-1] : 0; Fts5Iter *pNew = 0; Fts5SegIter *pNewIter = 0; Fts5SegIter *pPrevIter = 0; int iLvl, iSeg, ii; pNew = fts5MultiIterAlloc(p, pStruct->nSegment); if( pSmall ){ fts5BufferSet(&p->rc, &bSeek, pSmall->n, pSmall->p); fts5BufferAppendBlob(&p->rc, &bSeek, 1, (const u8*)"\0"); }else{ fts5BufferSet(&p->rc, &bSeek, nToken, pToken); } if( p->rc ){ sqlite3Fts5IterClose((Fts5IndexIter*)pNew); break; } pNewIter = &pNew->aSeg[0]; pPrevIter = (pPrev ? &pPrev->aSeg[0] : 0); for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=pStruct->aLevel[iLvl].nSeg-1; iSeg>=0; iSeg--){ Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg]; int bDone = 0; if( pPrevIter ){ if( fts5BufferCompare(pSmall, &pPrevIter->term) ){ memcpy(pNewIter, pPrevIter, sizeof(Fts5SegIter)); memset(pPrevIter, 0, sizeof(Fts5SegIter)); bDone = 1; }else if( pPrevIter->iEndofDoclist>pPrevIter->pLeaf->szLeaf ){ fts5SegIterNextInit(p,(const char*)bSeek.p,bSeek.n-1,pSeg,pNewIter); bDone = 1; } } if( bDone==0 ){ fts5SegIterSeekInit(p, bSeek.p, bSeek.n, flags, pSeg, pNewIter); } if( pPrevIter ){ if( pPrevIter->pTombArray ){ pNewIter->pTombArray = pPrevIter->pTombArray; pNewIter->pTombArray->nRef++; } }else{ fts5SegIterAllocTombstone(p, pNewIter); } pNewIter++; if( pPrevIter ) pPrevIter++; if( p->rc ) break; } } fts5TokendataSetTermIfEof(pPrev, pSmall); pNew->bSkipEmpty = 1; pNew->pColset = pColset; fts5IterSetOutputCb(&p->rc, pNew); /* Loop through all segments in the new iterator. Find the smallest ** term that any segment-iterator points to. Iterator pNew will be ** used for this term. Also, set any iterator that points to a term that ** does not match pToken/nToken to point to EOF */ pSmall = 0; for(ii=0; ii<pNew->nSeg; ii++){ Fts5SegIter *pII = &pNew->aSeg[ii]; if( 0==fts5IsTokendataPrefix(&pII->term, pToken, nToken) ){ fts5SegIterSetEOF(pII); } if( pII->pLeaf && (!pSmall || fts5BufferCompare(pSmall, &pII->term)>0) ){ pSmall = &pII->term; } } /* If pSmall is still NULL at this point, then the new iterator does ** not point to any terms that match the query. So delete it and break ** out of the loop - all required iterators have been collected. */ if( pSmall==0 ){ sqlite3Fts5IterClose((Fts5IndexIter*)pNew); break; } /* Append this iterator to the set and continue. */ pSet = fts5AppendTokendataIter(p, pSet, pNew); } if( p->rc==SQLITE_OK && pSet ){ int ii; for(ii=0; ii<pSet->nIter; ii++){ Fts5Iter *pIter = pSet->apIter[ii]; int iSeg; for(iSeg=0; iSeg<pIter->nSeg; iSeg++){ pIter->aSeg[iSeg].flags |= FTS5_SEGITER_ONETERM; } fts5MultiIterFinishSetup(p, pIter); } } if( p->rc==SQLITE_OK ){ pRet = fts5MultiIterAlloc(p, 0); } if( pRet ){ pRet->pTokenDataIter = pSet; if( pSet ){ fts5IterSetOutputsTokendata(pRet); }else{ pRet->base.bEof = 1; } }else{ fts5TokendataIterDelete(pSet); } fts5StructureRelease(pStruct); fts5BufferFree(&bSeek); return pRet; } /* ** Open a new iterator to iterate though all rowid that match the ** specified token or token prefix. */ static int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ |
︙ | ︙ | |||
242635 242636 242637 242638 242639 242640 242641 242642 242643 242644 242645 242646 242647 242648 242649 | /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ int iPrefixIdx = 0; /* +1 prefix index */ if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken); /* Figure out which index to search and set iIdx accordingly. If this ** is a prefix query for which there is no prefix index, set iIdx to ** greater than pConfig->nPrefix to indicate that the query will be ** satisfied by scanning multiple terms in the main index. ** ** If the QUERY_TEST_NOIDX flag was specified, then this must be a | > > > > > | 245516 245517 245518 245519 245520 245521 245522 245523 245524 245525 245526 245527 245528 245529 245530 245531 245532 245533 245534 245535 | /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ int iPrefixIdx = 0; /* +1 prefix index */ int bTokendata = pConfig->bTokendata; if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken); if( flags & (FTS5INDEX_QUERY_NOTOKENDATA|FTS5INDEX_QUERY_SCAN) ){ bTokendata = 0; } /* Figure out which index to search and set iIdx accordingly. If this ** is a prefix query for which there is no prefix index, set iIdx to ** greater than pConfig->nPrefix to indicate that the query will be ** satisfied by scanning multiple terms in the main index. ** ** If the QUERY_TEST_NOIDX flag was specified, then this must be a |
︙ | ︙ | |||
242662 242663 242664 242665 242666 242667 242668 | for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ int nIdxChar = pConfig->aPrefix[iIdx-1]; if( nIdxChar==nChar ) break; if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx; } } | > > > | | 245548 245549 245550 245551 245552 245553 245554 245555 245556 245557 245558 245559 245560 245561 245562 245563 245564 245565 | for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ int nIdxChar = pConfig->aPrefix[iIdx-1]; if( nIdxChar==nChar ) break; if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx; } } if( bTokendata && iIdx==0 ){ buf.p[0] = '0'; pRet = fts5SetupTokendataIter(p, buf.p, nToken+1, pColset); }else if( iIdx<=pConfig->nPrefix ){ /* Straight index lookup */ Fts5Structure *pStruct = fts5StructureRead(p); buf.p[0] = (u8)(FTS5_MAIN_PREFIX + iIdx); if( pStruct ){ fts5MultiIterNew(p, pStruct, flags | FTS5INDEX_QUERY_SKIPEMPTY, pColset, buf.p, nToken+1, -1, 0, &pRet ); |
︙ | ︙ | |||
242709 242710 242711 242712 242713 242714 242715 | */ /* ** Move to the next matching rowid. */ static int sqlite3Fts5IterNext(Fts5IndexIter *pIndexIter){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; assert( pIter->pIndex->rc==SQLITE_OK ); | > > > | > | 245598 245599 245600 245601 245602 245603 245604 245605 245606 245607 245608 245609 245610 245611 245612 245613 245614 245615 245616 | */ /* ** Move to the next matching rowid. */ static int sqlite3Fts5IterNext(Fts5IndexIter *pIndexIter){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; assert( pIter->pIndex->rc==SQLITE_OK ); if( pIter->pTokenDataIter ){ fts5TokendataIterNext(pIter, 0, 0); }else{ fts5MultiIterNext(pIter->pIndex, pIter, 0, 0); } return fts5IndexReturn(pIter->pIndex); } /* ** Move to the next matching term/rowid. Used by the fts5vocab module. */ static int sqlite3Fts5IterNextScan(Fts5IndexIter *pIndexIter){ |
︙ | ︙ | |||
242742 242743 242744 242745 242746 242747 242748 | /* ** Move to the next matching rowid that occurs at or after iMatch. The ** definition of "at or after" depends on whether this iterator iterates ** in ascending or descending rowid order. */ static int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; | > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 245635 245636 245637 245638 245639 245640 245641 245642 245643 245644 245645 245646 245647 245648 245649 245650 245651 245652 245653 245654 245655 245656 245657 245658 245659 245660 245661 245662 245663 245664 245665 245666 245667 245668 245669 245670 245671 245672 245673 245674 245675 245676 245677 245678 245679 245680 245681 245682 245683 245684 245685 245686 245687 245688 245689 245690 245691 245692 245693 245694 245695 245696 245697 245698 245699 245700 245701 245702 245703 245704 245705 245706 245707 245708 245709 245710 245711 245712 245713 245714 245715 245716 245717 245718 245719 245720 245721 245722 245723 245724 245725 245726 245727 245728 245729 245730 245731 245732 245733 245734 245735 245736 245737 245738 245739 245740 245741 245742 245743 245744 245745 245746 245747 245748 245749 245750 245751 245752 245753 245754 245755 245756 245757 245758 245759 245760 245761 245762 245763 245764 245765 245766 245767 245768 | /* ** Move to the next matching rowid that occurs at or after iMatch. The ** definition of "at or after" depends on whether this iterator iterates ** in ascending or descending rowid order. */ static int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; if( pIter->pTokenDataIter ){ fts5TokendataIterNext(pIter, 1, iMatch); }else{ fts5MultiIterNextFrom(pIter->pIndex, pIter, iMatch); } return fts5IndexReturn(pIter->pIndex); } /* ** Return the current term. */ static const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){ int n; const char *z = (const char*)fts5MultiIterTerm((Fts5Iter*)pIndexIter, &n); assert_nc( z || n<=1 ); *pn = n-1; return (z ? &z[1] : 0); } /* ** This is used by xInstToken() to access the token at offset iOff, column ** iCol of row iRowid. The token is returned via output variables *ppOut ** and *pnOut. The iterator passed as the first argument must be a tokendata=1 ** iterator (pIter->pTokenDataIter!=0). */ static int sqlite3Fts5IterToken( Fts5IndexIter *pIndexIter, i64 iRowid, int iCol, int iOff, const char **ppOut, int *pnOut ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5TokenDataIter *pT = pIter->pTokenDataIter; Fts5TokenDataMap *aMap = pT->aMap; i64 iPos = (((i64)iCol)<<32) + iOff; int i1 = 0; int i2 = pT->nMap; int iTest = 0; while( i2>i1 ){ iTest = (i1 + i2) / 2; if( aMap[iTest].iRowid<iRowid ){ i1 = iTest+1; }else if( aMap[iTest].iRowid>iRowid ){ i2 = iTest; }else{ if( aMap[iTest].iPos<iPos ){ if( aMap[iTest].iPos<0 ){ break; } i1 = iTest+1; }else if( aMap[iTest].iPos>iPos ){ i2 = iTest; }else{ break; } } } if( i2>i1 ){ Fts5Iter *pMap = pT->apIter[aMap[iTest].iIter]; *ppOut = (const char*)pMap->aSeg[0].term.p+1; *pnOut = pMap->aSeg[0].term.n-1; } return SQLITE_OK; } /* ** Clear any existing entries from the token-map associated with the ** iterator passed as the only argument. */ static void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter *pIndexIter){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; if( pIter && pIter->pTokenDataIter ){ pIter->pTokenDataIter->nMap = 0; } } /* ** Set a token-mapping for the iterator passed as the first argument. This ** is used in detail=column or detail=none mode when a token is requested ** using the xInstToken() API. In this case the caller tokenizers the ** current row and configures the token-mapping via multiple calls to this ** function. */ static int sqlite3Fts5IndexIterWriteTokendata( Fts5IndexIter *pIndexIter, const char *pToken, int nToken, i64 iRowid, int iCol, int iOff ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5TokenDataIter *pT = pIter->pTokenDataIter; Fts5Index *p = pIter->pIndex; int ii; assert( p->pConfig->eDetail!=FTS5_DETAIL_FULL ); assert( pIter->pTokenDataIter ); for(ii=0; ii<pT->nIter; ii++){ Fts5Buffer *pTerm = &pT->apIter[ii]->aSeg[0].term; if( nToken==pTerm->n-1 && memcmp(pToken, pTerm->p+1, nToken)==0 ) break; } if( ii<pT->nIter ){ fts5TokendataIterAppendMap(p, pT, ii, iRowid, (((i64)iCol)<<32) + iOff); } return fts5IndexReturn(p); } /* ** Close an iterator opened by an earlier call to sqlite3Fts5IndexQuery(). */ static void sqlite3Fts5IterClose(Fts5IndexIter *pIndexIter){ if( pIndexIter ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5Index *pIndex = pIter->pIndex; fts5TokendataIterDelete(pIter->pTokenDataIter); fts5MultiIterFree(pIter); sqlite3Fts5IndexCloseReader(pIndex); } } /* ** Read and decode the "averages" record from the database. |
︙ | ︙ | |||
243271 243272 243273 243274 243275 243276 243277 | int n, /* Size of index key in bytes */ int flags, /* Flags for Fts5IndexQuery */ u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; | | > > | 246262 246263 246264 246265 246266 246267 246268 246269 246270 246271 246272 246273 246274 246275 246276 246277 246278 | int n, /* Size of index key in bytes */ int flags, /* Flags for Fts5IndexQuery */ u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; int rc = sqlite3Fts5IndexQuery( p, z, n, (flags | FTS5INDEX_QUERY_NOTOKENDATA), 0, &pIter ); while( rc==SQLITE_OK && ALWAYS(pIter!=0) && 0==sqlite3Fts5IterEof(pIter) ){ i64 rowid = pIter->iRowid; if( eDetail==FTS5_DETAIL_NONE ){ cksum ^= sqlite3Fts5IndexEntryCksum(rowid, 0, 0, iIdx, z, n); }else{ |
︙ | ︙ | |||
243438 243439 243440 243441 243442 243443 243444 | if( i>=iNoRowid && 0!=fts5LeafFirstRowidOff(pLeaf) ) p->rc = FTS5_CORRUPT; } fts5DataRelease(pLeaf); } } static void fts5IntegrityCheckPgidx(Fts5Index *p, Fts5Data *pLeaf){ | | | | 246431 246432 246433 246434 246435 246436 246437 246438 246439 246440 246441 246442 246443 246444 246445 246446 246447 246448 246449 246450 246451 246452 246453 246454 | if( i>=iNoRowid && 0!=fts5LeafFirstRowidOff(pLeaf) ) p->rc = FTS5_CORRUPT; } fts5DataRelease(pLeaf); } } static void fts5IntegrityCheckPgidx(Fts5Index *p, Fts5Data *pLeaf){ i64 iTermOff = 0; int ii; Fts5Buffer buf1 = {0,0,0}; Fts5Buffer buf2 = {0,0,0}; ii = pLeaf->szLeaf; while( ii<pLeaf->nn && p->rc==SQLITE_OK ){ int res; i64 iOff; int nIncr; ii += fts5GetVarint32(&pLeaf->p[ii], nIncr); iTermOff += nIncr; iOff = iTermOff; if( iOff>=pLeaf->szLeaf ){ |
︙ | ︙ | |||
243968 243969 243970 243971 243972 243973 243974 243975 243976 243977 243978 243979 243980 243981 | } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** The implementation of user-defined scalar function fts5_decode(). */ static void fts5DecodeFunction( sqlite3_context *pCtx, /* Function call context */ | > > > > > > > > > > > > > > > > > > | 246961 246962 246963 246964 246965 246966 246967 246968 246969 246970 246971 246972 246973 246974 246975 246976 246977 246978 246979 246980 246981 246982 246983 246984 246985 246986 246987 246988 246989 246990 246991 246992 | } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) static void fts5BufferAppendTerm(int *pRc, Fts5Buffer *pBuf, Fts5Buffer *pTerm){ int ii; fts5BufferGrow(pRc, pBuf, pTerm->n*2 + 1); if( *pRc==SQLITE_OK ){ for(ii=0; ii<pTerm->n; ii++){ if( pTerm->p[ii]==0x00 ){ pBuf->p[pBuf->n++] = '\\'; pBuf->p[pBuf->n++] = '0'; }else{ pBuf->p[pBuf->n++] = pTerm->p[ii]; } } pBuf->p[pBuf->n] = 0x00; } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** The implementation of user-defined scalar function fts5_decode(). */ static void fts5DecodeFunction( sqlite3_context *pCtx, /* Function call context */ |
︙ | ︙ | |||
244076 244077 244078 244079 244080 244081 244082 | while( iOff<szLeaf && rc==SQLITE_OK ){ int nAppend; /* Read the term data for the next term*/ iOff += fts5GetVarint32(&a[iOff], nAppend); term.n = nKeep; fts5BufferAppendBlob(&rc, &term, nAppend, &a[iOff]); | | | < | 247087 247088 247089 247090 247091 247092 247093 247094 247095 247096 247097 247098 247099 247100 247101 247102 | while( iOff<szLeaf && rc==SQLITE_OK ){ int nAppend; /* Read the term data for the next term*/ iOff += fts5GetVarint32(&a[iOff], nAppend); term.n = nKeep; fts5BufferAppendBlob(&rc, &term, nAppend, &a[iOff]); sqlite3Fts5BufferAppendPrintf(&rc, &s, " term="); fts5BufferAppendTerm(&rc, &s, &term); iOff += nAppend; /* Figure out where the doclist for this term ends */ if( iPgidxOff<n ){ int nIncr; iPgidxOff += fts5GetVarint32(&a[iPgidxOff], nIncr); iTermOff += nIncr; |
︙ | ︙ | |||
244186 244187 244188 244189 244190 244191 244192 | if( iOff+nByte>n ){ rc = FTS5_CORRUPT; break; } fts5BufferAppendBlob(&rc, &term, nByte, &a[iOff]); iOff += nByte; | | | < | 247196 247197 247198 247199 247200 247201 247202 247203 247204 247205 247206 247207 247208 247209 247210 247211 | if( iOff+nByte>n ){ rc = FTS5_CORRUPT; break; } fts5BufferAppendBlob(&rc, &term, nByte, &a[iOff]); iOff += nByte; sqlite3Fts5BufferAppendPrintf(&rc, &s, " term="); fts5BufferAppendTerm(&rc, &s, &term); iOff += fts5DecodeDoclist(&rc, &s, &a[iOff], iEnd-iOff); } fts5BufferFree(&term); } decode_out: |
︙ | ︙ | |||
244663 244664 244665 244666 244667 244668 244669 | struct Fts5FullTable { Fts5Table p; /* Public class members from fts5Int.h */ Fts5Storage *pStorage; /* Document store */ Fts5Global *pGlobal; /* Global (connection wide) data */ Fts5Cursor *pSortCsr; /* Sort data from this cursor */ int iSavepoint; /* Successful xSavepoint()+1 */ | | | 247672 247673 247674 247675 247676 247677 247678 247679 247680 247681 247682 247683 247684 247685 247686 | struct Fts5FullTable { Fts5Table p; /* Public class members from fts5Int.h */ Fts5Storage *pStorage; /* Document store */ Fts5Global *pGlobal; /* Global (connection wide) data */ Fts5Cursor *pSortCsr; /* Sort data from this cursor */ int iSavepoint; /* Successful xSavepoint()+1 */ #ifdef SQLITE_DEBUG struct Fts5TransactionState ts; #endif }; struct Fts5MatchPhrase { Fts5Buffer *pPoslist; /* Pointer to current poslist */ |
︙ | ︙ | |||
245201 245202 245203 245204 245205 245206 245207 | bSeenGt = 1; } } } } idxStr[iIdxStr] = '\0'; | | > > > | | 248210 248211 248212 248213 248214 248215 248216 248217 248218 248219 248220 248221 248222 248223 248224 248225 248226 248227 248228 248229 248230 248231 248232 | bSeenGt = 1; } } } } idxStr[iIdxStr] = '\0'; /* Set idxFlags flags for the ORDER BY clause ** ** Note that tokendata=1 tables cannot currently handle "ORDER BY rowid DESC". */ if( pInfo->nOrderBy==1 ){ int iSort = pInfo->aOrderBy[0].iColumn; if( iSort==(pConfig->nCol+1) && bSeenMatch ){ idxFlags |= FTS5_BI_ORDER_RANK; }else if( iSort==-1 && (!pInfo->aOrderBy[0].desc || !pConfig->bTokendata) ){ idxFlags |= FTS5_BI_ORDER_ROWID; } if( BitFlagTest(idxFlags, FTS5_BI_ORDER_RANK|FTS5_BI_ORDER_ROWID) ){ pInfo->orderByConsumed = 1; if( pInfo->aOrderBy[0].desc ){ idxFlags |= FTS5_BI_ORDER_DESC; } |
︙ | ︙ | |||
245457 245458 245459 245460 245461 245462 245463 245464 245465 245466 245467 245468 245469 245470 | Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc; assert( (pCsr->ePlan<3)== (pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE) ); assert( !CsrFlagTest(pCsr, FTS5CSR_EOF) ); if( pCsr->ePlan<3 ){ int bSkip = 0; if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc; rc = sqlite3Fts5ExprNext(pCsr->pExpr, pCsr->iLastRowid); CsrFlagSet(pCsr, sqlite3Fts5ExprEof(pCsr->pExpr)); fts5CsrNewrow(pCsr); | > > > > > > > > > > | 248469 248470 248471 248472 248473 248474 248475 248476 248477 248478 248479 248480 248481 248482 248483 248484 248485 248486 248487 248488 248489 248490 248491 248492 | Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc; assert( (pCsr->ePlan<3)== (pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE) ); assert( !CsrFlagTest(pCsr, FTS5CSR_EOF) ); /* If this cursor uses FTS5_PLAN_MATCH and this is a tokendata=1 table, ** clear any token mappings accumulated at the fts5_index.c level. In ** other cases, specifically FTS5_PLAN_SOURCE and FTS5_PLAN_SORTED_MATCH, ** we need to retain the mappings for the entire query. */ if( pCsr->ePlan==FTS5_PLAN_MATCH && ((Fts5Table*)pCursor->pVtab)->pConfig->bTokendata ){ sqlite3Fts5ExprClearTokens(pCsr->pExpr); } if( pCsr->ePlan<3 ){ int bSkip = 0; if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc; rc = sqlite3Fts5ExprNext(pCsr->pExpr, pCsr->iLastRowid); CsrFlagSet(pCsr, sqlite3Fts5ExprEof(pCsr->pExpr)); fts5CsrNewrow(pCsr); |
︙ | ︙ | |||
246118 246119 246120 246121 246122 246123 246124 | #ifdef SQLITE_DEBUG }else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){ pConfig->bPrefixIndex = sqlite3_value_int(pVal); #endif }else if( 0==sqlite3_stricmp("flush", zCmd) ){ rc = sqlite3Fts5FlushToDisk(&pTab->p); }else{ | > > | > | 249140 249141 249142 249143 249144 249145 249146 249147 249148 249149 249150 249151 249152 249153 249154 249155 249156 249157 | #ifdef SQLITE_DEBUG }else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){ pConfig->bPrefixIndex = sqlite3_value_int(pVal); #endif }else if( 0==sqlite3_stricmp("flush", zCmd) ){ rc = sqlite3Fts5FlushToDisk(&pTab->p); }else{ rc = sqlite3Fts5FlushToDisk(&pTab->p); if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigSetValue(pTab->p.pConfig, zCmd, pVal, &bError); } if( rc==SQLITE_OK ){ if( bError ){ rc = SQLITE_ERROR; }else{ |
︙ | ︙ | |||
246443 246444 246445 246446 246447 246448 246449 | Fts5Context *pCtx, int iCol, const char **pz, int *pn ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | > > > | | 249468 249469 249470 249471 249472 249473 249474 249475 249476 249477 249478 249479 249480 249481 249482 249483 249484 249485 | Fts5Context *pCtx, int iCol, const char **pz, int *pn ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab); if( iCol<0 || iCol>=pTab->pConfig->nCol ){ rc = SQLITE_RANGE; }else if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab)) || pCsr->ePlan==FTS5_PLAN_SPECIAL ){ *pz = 0; *pn = 0; }else{ rc = fts5SeekCursor(pCsr, 0); if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
246468 246469 246470 246471 246472 246473 246474 | const u8 **pa, int *pn ){ Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig; int rc = SQLITE_OK; int bLive = (pCsr->pSorter==0); | > > | < | 249496 249497 249498 249499 249500 249501 249502 249503 249504 249505 249506 249507 249508 249509 249510 249511 249512 | const u8 **pa, int *pn ){ Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig; int rc = SQLITE_OK; int bLive = (pCsr->pSorter==0); if( iPhrase<0 || iPhrase>=sqlite3Fts5ExprPhraseCount(pCsr->pExpr) ){ rc = SQLITE_RANGE; }else if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){ if( pConfig->eDetail!=FTS5_DETAIL_FULL ){ Fts5PoslistPopulator *aPopulator; int i; aPopulator = sqlite3Fts5ExprClearPoslists(pCsr->pExpr, bLive); if( aPopulator==0 ) rc = SQLITE_NOMEM; for(i=0; i<pConfig->nCol && rc==SQLITE_OK; i++){ int n; const char *z; |
︙ | ︙ | |||
246493 246494 246495 246496 246497 246498 246499 | if( pCsr->pSorter ){ sqlite3Fts5ExprCheckPoslists(pCsr->pExpr, pCsr->pSorter->iRowid); } } CsrFlagClear(pCsr, FTS5CSR_REQUIRE_POSLIST); } | > | | | | | | | | > > > > > | 249522 249523 249524 249525 249526 249527 249528 249529 249530 249531 249532 249533 249534 249535 249536 249537 249538 249539 249540 249541 249542 249543 249544 249545 249546 249547 249548 249549 | if( pCsr->pSorter ){ sqlite3Fts5ExprCheckPoslists(pCsr->pExpr, pCsr->pSorter->iRowid); } } CsrFlagClear(pCsr, FTS5CSR_REQUIRE_POSLIST); } if( rc==SQLITE_OK ){ if( pCsr->pSorter && pConfig->eDetail==FTS5_DETAIL_FULL ){ Fts5Sorter *pSorter = pCsr->pSorter; int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]); *pn = pSorter->aIdx[iPhrase] - i1; *pa = &pSorter->aPoslist[i1]; }else{ *pn = sqlite3Fts5ExprPoslist(pCsr->pExpr, iPhrase, pa); } }else{ *pa = 0; *pn = 0; } return rc; } /* ** Ensure that the Fts5Cursor.nInstCount and aInst[] variables are populated ** correctly for the current view. Return SQLITE_OK if successful, or an |
︙ | ︙ | |||
246608 246609 246610 246611 246612 246613 246614 | Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; | < < < < < < | 249643 249644 249645 249646 249647 249648 249649 249650 249651 249652 249653 249654 249655 249656 | Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; }else{ *piPhrase = pCsr->aInst[iIdx*3]; *piCol = pCsr->aInst[iIdx*3 + 1]; *piOff = pCsr->aInst[iIdx*3 + 2]; } } return rc; |
︙ | ︙ | |||
246868 246869 246870 246871 246872 246873 246874 246875 246876 246877 246878 246879 246880 | } } } return rc; } static int fts5ApiQueryPhrase(Fts5Context*, int, void*, int(*)(const Fts5ExtensionApi*, Fts5Context*, void*) ); static const Fts5ExtensionApi sFts5Api = { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > | 249897 249898 249899 249900 249901 249902 249903 249904 249905 249906 249907 249908 249909 249910 249911 249912 249913 249914 249915 249916 249917 249918 249919 249920 249921 249922 249923 249924 249925 249926 249927 249928 249929 249930 249931 249932 249933 249934 249935 249936 249937 249938 249939 249940 249941 249942 249943 249944 249945 249946 249947 249948 249949 249950 249951 249952 249953 249954 249955 249956 249957 249958 249959 249960 249961 249962 249963 249964 249965 249966 249967 249968 249969 249970 249971 249972 249973 249974 249975 249976 249977 249978 249979 249980 249981 | } } } return rc; } /* ** xQueryToken() API implemenetation. */ static int fts5ApiQueryToken( Fts5Context* pCtx, int iPhrase, int iToken, const char **ppOut, int *pnOut ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; return sqlite3Fts5ExprQueryToken(pCsr->pExpr, iPhrase, iToken, ppOut, pnOut); } /* ** xInstToken() API implemenetation. */ static int fts5ApiInstToken( Fts5Context *pCtx, int iIdx, int iToken, const char **ppOut, int *pnOut ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; }else{ int iPhrase = pCsr->aInst[iIdx*3]; int iCol = pCsr->aInst[iIdx*3 + 1]; int iOff = pCsr->aInst[iIdx*3 + 2]; i64 iRowid = fts5CursorRowid(pCsr); rc = sqlite3Fts5ExprInstToken( pCsr->pExpr, iRowid, iPhrase, iCol, iOff, iToken, ppOut, pnOut ); } } return rc; } static int fts5ApiQueryPhrase(Fts5Context*, int, void*, int(*)(const Fts5ExtensionApi*, Fts5Context*, void*) ); static const Fts5ExtensionApi sFts5Api = { 3, /* iVersion */ fts5ApiUserData, fts5ApiColumnCount, fts5ApiRowCount, fts5ApiColumnTotalSize, fts5ApiTokenize, fts5ApiPhraseCount, fts5ApiPhraseSize, fts5ApiInstCount, fts5ApiInst, fts5ApiRowid, fts5ApiColumnText, fts5ApiColumnSize, fts5ApiQueryPhrase, fts5ApiSetAuxdata, fts5ApiGetAuxdata, fts5ApiPhraseFirst, fts5ApiPhraseNext, fts5ApiPhraseFirstColumn, fts5ApiPhraseNextColumn, fts5ApiQueryToken, fts5ApiInstToken }; /* ** Implementation of API function xQueryPhrase(). */ static int fts5ApiQueryPhrase( Fts5Context *pCtx, |
︙ | ︙ | |||
247160 247161 247162 247163 247164 247165 247166 | */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; | < < | | < < < < < < < < < < < < | | | < < | 250234 250235 250236 250237 250238 250239 250240 250241 250242 250243 250244 250245 250246 250247 250248 250249 250250 250251 250252 250253 250254 250255 250256 250257 250258 250259 250260 250261 250262 250263 250264 250265 250266 250267 250268 250269 250270 | */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; rc = sqlite3Fts5StorageRename(pTab->pStorage, zName); return rc; } static int sqlite3Fts5FlushToDisk(Fts5Table *pTab){ fts5TripCursors((Fts5FullTable*)pTab); return sqlite3Fts5StorageSync(((Fts5FullTable*)pTab)->pStorage); } /* ** The xSavepoint() method. ** ** Flush the contents of the pending-terms table to disk. */ static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_SAVEPOINT, iSavepoint); rc = sqlite3Fts5FlushToDisk((Fts5Table*)pVtab); if( rc==SQLITE_OK ){ pTab->iSavepoint = iSavepoint+1; } return rc; } /* ** The xRelease() method. ** ** This is a no-op. |
︙ | ︙ | |||
247230 247231 247232 247233 247234 247235 247236 | ** Discard the contents of the pending terms table. */ static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint); fts5TripCursors(pTab); | < > | 250288 250289 250290 250291 250292 250293 250294 250295 250296 250297 250298 250299 250300 250301 250302 250303 | ** Discard the contents of the pending terms table. */ static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint); fts5TripCursors(pTab); if( (iSavepoint+1)<=pTab->iSavepoint ){ pTab->p.pConfig->pgsz = 0; rc = sqlite3Fts5StorageRollback(pTab->pStorage); } return rc; } /* ** Register a new auxiliary function with global context pGlobal. |
︙ | ︙ | |||
247436 247437 247438 247439 247440 247441 247442 | static void fts5SourceIdFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); | | | 250494 250495 250496 250497 250498 250499 250500 250501 250502 250503 250504 250505 250506 250507 250508 | static void fts5SourceIdFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); sqlite3_result_text(pCtx, "fts5: 2024-01-12 11:44:49 f47a5f4e0ce078e6cc1183e6cbb3c4013af379b496efae94863a42e5c39928ed", -1, SQLITE_TRANSIENT); } /* ** Return true if zName is the extension on one of the shadow tables used ** by this module. */ static int fts5ShadowName(const char *zName){ |
︙ | ︙ | |||
247459 247460 247461 247462 247463 247464 247465 | } /* ** Run an integrity check on the FTS5 data structures. Return a string ** if anything is found amiss. Return a NULL pointer if everything is ** OK. */ | | > > > > > > > > | | | | 250517 250518 250519 250520 250521 250522 250523 250524 250525 250526 250527 250528 250529 250530 250531 250532 250533 250534 250535 250536 250537 250538 250539 250540 250541 250542 250543 250544 250545 250546 250547 250548 250549 250550 250551 250552 250553 250554 250555 250556 250557 | } /* ** Run an integrity check on the FTS5 data structures. Return a string ** if anything is found amiss. Return a NULL pointer if everything is ** OK. */ static int fts5IntegrityMethod( sqlite3_vtab *pVtab, /* the FTS5 virtual table to check */ const char *zSchema, /* Name of schema in which this table lives */ const char *zTabname, /* Name of the table itself */ int isQuick, /* True if this is a quick-check */ char **pzErr /* Write error message here */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; Fts5Config *pConfig = pTab->p.pConfig; char *zSql; char *zErr = 0; int rc; assert( pzErr!=0 && *pzErr==0 ); UNUSED_PARAM(isQuick); zSql = sqlite3_mprintf( "INSERT INTO \"%w\".\"%w\"(\"%w\") VALUES('integrity-check');", zSchema, zTabname, pConfig->zName); if( zSql==0 ) return SQLITE_NOMEM; rc = sqlite3_exec(pConfig->db, zSql, 0, 0, &zErr); sqlite3_free(zSql); if( (rc&0xff)==SQLITE_CORRUPT ){ *pzErr = sqlite3_mprintf("malformed inverted index for FTS5 table %s.%s", zSchema, zTabname); }else if( rc!=SQLITE_OK ){ *pzErr = sqlite3_mprintf("unable to validate the inverted index for" " FTS5 table %s.%s: %s", zSchema, zTabname, zErr); } sqlite3_free(zErr); return SQLITE_OK; } static int fts5Init(sqlite3 *db){ static const sqlite3_module fts5Mod = { |
︙ | ︙ | |||
247509 247510 247511 247512 247513 247514 247515 | /* xRollback */ fts5RollbackMethod, /* xFindFunction */ fts5FindFunctionMethod, /* xRename */ fts5RenameMethod, /* xSavepoint */ fts5SavepointMethod, /* xRelease */ fts5ReleaseMethod, /* xRollbackTo */ fts5RollbackToMethod, /* xShadowName */ fts5ShadowName, | | | 250575 250576 250577 250578 250579 250580 250581 250582 250583 250584 250585 250586 250587 250588 250589 | /* xRollback */ fts5RollbackMethod, /* xFindFunction */ fts5FindFunctionMethod, /* xRename */ fts5RenameMethod, /* xSavepoint */ fts5SavepointMethod, /* xRelease */ fts5ReleaseMethod, /* xRollbackTo */ fts5RollbackToMethod, /* xShadowName */ fts5ShadowName, /* xIntegrity */ fts5IntegrityMethod }; int rc; Fts5Global *pGlobal = 0; pGlobal = (Fts5Global*)sqlite3_malloc(sizeof(Fts5Global)); if( pGlobal==0 ){ |
︙ | ︙ | |||
248275 248276 248277 248278 248279 248280 248281 | ctx.pStorage = p; rc = sqlite3Fts5StorageDeleteAll(p); if( rc==SQLITE_OK ){ rc = fts5StorageLoadTotals(p, 1); } if( rc==SQLITE_OK ){ | | | 251341 251342 251343 251344 251345 251346 251347 251348 251349 251350 251351 251352 251353 251354 251355 | ctx.pStorage = p; rc = sqlite3Fts5StorageDeleteAll(p); if( rc==SQLITE_OK ){ rc = fts5StorageLoadTotals(p, 1); } if( rc==SQLITE_OK ){ rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, pConfig->pzErrmsg); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pScan) ){ i64 iRowid = sqlite3_column_int64(pScan, 0); sqlite3Fts5BufferZero(&buf); rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); |
︙ | ︙ | |||
249061 249062 249063 249064 249065 249066 249067 249068 249069 249070 249071 249072 249073 249074 | *zOut++ = 0x80 + (unsigned char)((c>>12) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)((c>>6) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)(c & 0x3F); \ } \ } #endif /* ifndef SQLITE_AMALGAMATION */ typedef struct Unicode61Tokenizer Unicode61Tokenizer; struct Unicode61Tokenizer { unsigned char aTokenChar[128]; /* ASCII range token characters */ char *aFold; /* Buffer to fold text into */ int nFold; /* Size of aFold[] in bytes */ int eRemoveDiacritic; /* True if remove_diacritics=1 is set */ | > > > > > > | 252127 252128 252129 252130 252131 252132 252133 252134 252135 252136 252137 252138 252139 252140 252141 252142 252143 252144 252145 252146 | *zOut++ = 0x80 + (unsigned char)((c>>12) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)((c>>6) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)(c & 0x3F); \ } \ } #endif /* ifndef SQLITE_AMALGAMATION */ #define FTS5_SKIP_UTF8(zIn) { \ if( ((unsigned char)(*(zIn++)))>=0xc0 ){ \ while( (((unsigned char)*zIn) & 0xc0)==0x80 ){ zIn++; } \ } \ } typedef struct Unicode61Tokenizer Unicode61Tokenizer; struct Unicode61Tokenizer { unsigned char aTokenChar[128]; /* ASCII range token characters */ char *aFold; /* Buffer to fold text into */ int nFold; /* Size of aFold[] in bytes */ int eRemoveDiacritic; /* True if remove_diacritics=1 is set */ |
︙ | ︙ | |||
250097 250098 250099 250100 250101 250102 250103 250104 250105 250106 250107 250108 250109 250110 | /************************************************************************** ** Start of trigram implementation. */ typedef struct TrigramTokenizer TrigramTokenizer; struct TrigramTokenizer { int bFold; /* True to fold to lower-case */ }; /* ** Free a trigram tokenizer. */ static void fts5TriDelete(Fts5Tokenizer *p){ sqlite3_free(p); | > | 253169 253170 253171 253172 253173 253174 253175 253176 253177 253178 253179 253180 253181 253182 253183 | /************************************************************************** ** Start of trigram implementation. */ typedef struct TrigramTokenizer TrigramTokenizer; struct TrigramTokenizer { int bFold; /* True to fold to lower-case */ int iFoldParam; /* Parameter to pass to Fts5UnicodeFold() */ }; /* ** Free a trigram tokenizer. */ static void fts5TriDelete(Fts5Tokenizer *p){ sqlite3_free(p); |
︙ | ︙ | |||
250123 250124 250125 250126 250127 250128 250129 250130 250131 250132 250133 250134 250135 250136 250137 250138 250139 250140 250141 250142 250143 250144 250145 250146 250147 250148 | TrigramTokenizer *pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew)); UNUSED_PARAM(pUnused); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ int i; pNew->bFold = 1; for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ const char *zArg = azArg[i+1]; if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->bFold = (zArg[0]=='0'); } }else{ rc = SQLITE_ERROR; } } if( rc!=SQLITE_OK ){ fts5TriDelete((Fts5Tokenizer*)pNew); pNew = 0; } } *ppOut = (Fts5Tokenizer*)pNew; return rc; | > > > > > > > > > > > > | 253196 253197 253198 253199 253200 253201 253202 253203 253204 253205 253206 253207 253208 253209 253210 253211 253212 253213 253214 253215 253216 253217 253218 253219 253220 253221 253222 253223 253224 253225 253226 253227 253228 253229 253230 253231 253232 253233 | TrigramTokenizer *pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew)); UNUSED_PARAM(pUnused); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ int i; pNew->bFold = 1; pNew->iFoldParam = 0; for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ const char *zArg = azArg[i+1]; if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->bFold = (zArg[0]=='0'); } }else if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){ if( (zArg[0]!='0' && zArg[0]!='1' && zArg[0]!='2') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->iFoldParam = (zArg[0]!='0') ? 2 : 0; } }else{ rc = SQLITE_ERROR; } } if( pNew->iFoldParam!=0 && pNew->bFold==0 ){ rc = SQLITE_ERROR; } if( rc!=SQLITE_OK ){ fts5TriDelete((Fts5Tokenizer*)pNew); pNew = 0; } } *ppOut = (Fts5Tokenizer*)pNew; return rc; |
︙ | ︙ | |||
250157 250158 250159 250160 250161 250162 250163 250164 250165 250166 250167 250168 250169 | int unusedFlags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; int rc = SQLITE_OK; char aBuf[32]; const unsigned char *zIn = (const unsigned char*)pText; const unsigned char *zEof = &zIn[nText]; u32 iCode; UNUSED_PARAM(unusedFlags); while( 1 ){ | > > > > > > > > > > > > > > > > > > > > > > > > | | < | > > | | < < | | | > > | < < | | | > | | < | | < < | > | < > | | 253242 253243 253244 253245 253246 253247 253248 253249 253250 253251 253252 253253 253254 253255 253256 253257 253258 253259 253260 253261 253262 253263 253264 253265 253266 253267 253268 253269 253270 253271 253272 253273 253274 253275 253276 253277 253278 253279 253280 253281 253282 253283 253284 253285 253286 253287 253288 253289 253290 253291 253292 253293 253294 253295 253296 253297 253298 253299 253300 253301 253302 253303 253304 253305 253306 253307 253308 253309 253310 253311 | int unusedFlags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; int rc = SQLITE_OK; char aBuf[32]; char *zOut = aBuf; int ii; const unsigned char *zIn = (const unsigned char*)pText; const unsigned char *zEof = &zIn[nText]; u32 iCode; int aStart[3]; /* Input offset of each character in aBuf[] */ UNUSED_PARAM(unusedFlags); /* Populate aBuf[] with the characters for the first trigram. */ for(ii=0; ii<3; ii++){ do { aStart[ii] = zIn - (const unsigned char*)pText; READ_UTF8(zIn, zEof, iCode); if( iCode==0 ) return SQLITE_OK; if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam); }while( iCode==0 ); WRITE_UTF8(zOut, iCode); } /* At the start of each iteration of this loop: ** ** aBuf: Contains 3 characters. The 3 characters of the next trigram. ** zOut: Points to the byte following the last character in aBuf. ** aStart[3]: Contains the byte offset in the input text corresponding ** to the start of each of the three characters in the buffer. */ assert( zIn<=zEof ); while( 1 ){ int iNext; /* Start of character following current tri */ const char *z1; /* Read characters from the input up until the first non-diacritic */ do { iNext = zIn - (const unsigned char*)pText; READ_UTF8(zIn, zEof, iCode); if( iCode==0 ) break; if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam); }while( iCode==0 ); /* Pass the current trigram back to fts5 */ rc = xToken(pCtx, 0, aBuf, zOut-aBuf, aStart[0], iNext); if( iCode==0 || rc!=SQLITE_OK ) break; /* Remove the first character from buffer aBuf[]. Append the character ** with codepoint iCode. */ z1 = aBuf; FTS5_SKIP_UTF8(z1); memmove(aBuf, z1, zOut - z1); zOut -= (z1 - aBuf); WRITE_UTF8(zOut, iCode); /* Update the aStart[] array */ aStart[0] = aStart[1]; aStart[1] = aStart[2]; aStart[2] = iNext; } return rc; } /* ** Argument xCreate is a pointer to a constructor function for a tokenizer. |
︙ | ︙ | |||
250213 250214 250215 250216 250217 250218 250219 | */ static int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ){ if( xCreate==fts5TriCreate ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; | > | > | 253320 253321 253322 253323 253324 253325 253326 253327 253328 253329 253330 253331 253332 253333 253334 253335 253336 | */ static int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ){ if( xCreate==fts5TriCreate ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; if( p->iFoldParam==0 ){ return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB; } } return FTS5_PATTERN_NONE; } /* ** Register all built-in tokenizers with FTS5. */ |
︙ | ︙ | |||
252002 252003 252004 252005 252006 252007 252008 | if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++]; if( pEq ){ zTerm = (const char *)sqlite3_value_text(pEq); nTerm = sqlite3_value_bytes(pEq); | | | 255111 255112 255113 255114 255115 255116 255117 255118 255119 255120 255121 255122 255123 255124 255125 | if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++]; if( pEq ){ zTerm = (const char *)sqlite3_value_text(pEq); nTerm = sqlite3_value_bytes(pEq); f = FTS5INDEX_QUERY_NOTOKENDATA; }else{ if( pGe ){ zTerm = (const char *)sqlite3_value_text(pGe); nTerm = sqlite3_value_bytes(pGe); } if( pLe ){ const char *zCopy = (const char *)sqlite3_value_text(pLe); |
︙ | ︙ |