Subject: [media-types] Proposed registration for application/vnd.sqlite3 and +sqlite3

From: Clemens Ladisch <[email-masked]>
Date: Sun, 1 Oct 2017 17:27:24 +0200

Type name:

  application

Subtype name:

  vnd.sqlite3

Required parameters:

  none

Optional parameters:

  none

Encoding considerations:

  binary

Security considerations:

  Database files contain complex data structures, so parsers must take
  care to prevent buffer overflows, stack overruns, and other unexpected
  behaviour caused by malicious content.

  Views and triggers can contain arbitrary SQL expressions (including
  recursion), which can result in arbitrarily large amounts of
  processing time, memory, and disk space required when attempting to
  access data.  Applications should use mechanisms like
  sqlite3_interrupt() or sqlite3_progress_handler() to allow long
  computations to be aborted, and an alternative memory allocator to
  limit the amount of memory used.

  The SQLite library itself, as distributed, does not allow SQL
  statements to access resources or data outside the database.  However,
  if applications add extension modules or functions, they should not do
  so in the database connection used to access untrusted content, or
  they must ensure that these modules/functions are safe to use even
  when called from malicious SQL code.

  The database may leave part of deleted or updated data in the database
  file.  Applications that do not want ot leave traces of old data must
  enable PRAGMA secure_delete before doing any modifications, or run
  VACUUM before transmitting the database file.

  Databases can use indexes to cache data in a format that is faster to
  access for certain queries.  It is possible to construct database
  files with inconsistent data in indexes so that some queries return
  data different from what is actually stored in a table.  To avoid
  this, applications should run REINDEX before accessing a database
  received from a potentially malicious source.

  This format provides no cryptographic integrity protection of any
  kind.

  Databases can be used to store blobs containing data to be handled by
  other applications or libraries; any security considerations of those
  must also be taken into account.

Interoperability considerations:

  At publication of this document, there exists only a single
  implementation, the SQLite library.

  Database files written with recent versions of the library can be read
  and modified by any version back to at least 3.7.0 (released
  2010-07-21).  However, there is no backwards compatibility if SQL
  features introduced in a newer version are actually used.  To ensure
  interoperability with other applications that use an older version of
  the library, applications SHOULD avoid using features that are not
  supported in the version that other applications are known or
  suspected to use.  At publication of this document, features
  introduced in newer versions are:

  3.20.0: deterministic date/time functions;
  3.18.0: printf() thousands marks;
  3.16.0: PRAGMA functions;
  3.15.0: row values; deterministic SQL functions in partial indexes;
  3.9.0: expression indexes;
  3.8.8: more than 500 rows in a VALUES clause;
  3.8.6: hexadecimal integer literals; likely();
  3.8.3: common table expressions (WITH); printf();
  3.8.2: clustered indexes (WITHOUT ROWID tables);
  3.8.1: unlikely(); likelihood();
  3.8.0: partial indexes;
  3.7.16: unicode(); char();
  3.7.15: instr();
  3.7.11: multiple rows in a VALUES clause; bare columns in aggregate queries.

  Some runtime settings (e.g., PRAGMA case_sensitive_like) or
  compilation options can change the semantics of SQL statements.
  Applications SHOULD use the default settings and options; however,
  some settings (e.g., PRAGMA foreign_keys) are disabled by default only
  for backwards compatibility and are commonly enabled.

  When a transaction that changes the database has not yet committed,
  the database file might be in an inconsistent state and require data
  from the rollback journal to get back to a consistent state.
  Therefore, when it is possible that other processes or threads change
  a database, an application that wishes to transmit a database file
  SHOULD prevent concurrent changes by executing BEGIN IMMEDIATE before
  reading/copying the file, or use the backup API to create a consistent
  copy of the database.

  A database in WAL mode can have part of its data in the WAL file.
  Therefore, an application that wishes to transmit a database file in
  WAL mode SHOULD initiate a full checkpoint before reading/copying the
  file, or use the backup API to create a copy of the database.

  The unregistered media type "application/x-sqlite3" MUST NOT be used,
  except where required for backwards compatibility.

Published specification:

  http://www.sqlite.org/fileformat2.html
  http://www.sqlite.org/lang.html

Applications that use this media type:

  Applications that want to store or interchange relational data.

Fragment identifier considerations:

  none

Deprecated alias names for this type:

  application/x-sqlite3

Magic number:

  53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00
  (zero-terminated ASCII "SQLite format 3") at offset 0

File extensions:

  .db, .sqlite, .sqlite3
  (".db" does not uniquely identify SQLite database files.
  Other extensions are commonly used.)

Macintosh file type code:

  none

Contact:

  SQLite mailing list
  <[email-masked]>

Intended usage:

  COMMON

Restrictions on usage:

  none

Author/Change controller:

  Clemens Ladisch
  <[email-masked]>

Provisional registration? (standards tree only):

  N/A

========================================================================

Name

  SQLite3 database

+suffix

  +sqlite3

References

  Same as for "application/vnd.sqlite3".

Encoding considerations

  binary

Interoperability considerations

  Same as for "application/vnd.sqlite3".

  To allow identification of files when the media type name is not
  available, each individual "xxx/yyy+sqlite3" registration SHOULD
  specify an appliction ID value to be set with PRAGMA application_id
  (http://www.sqlite.org/pragma.html#pragma_application_id), and SHOULD
  specifiy it as a second magic number (file offset 68, see
  http://www.sqlite.org/fileformat2.html#application_id) in addition to
  the header string at offset 0.  This value should also be added to the
  magic.txt file in the SQLite repository
  (http://www.sqlite.org/src/artifact?ci=trunk&filename=magic.txt) by
  submitting a patch to <[email-masked]>.

Fragment identifier considerations

  The syntax and semantics of fragment identifiers specified for
  +sqlite3 SHOULD be as specified for "application/vnd.sqlite3".
  (At publication of this document, there is no fragment identification
  syntax defined for "application/vnd.sqlite3".)

  The syntax and semantics of fragment identifiers for a specific
  "xxx/yyy+sqlite3" SHOULD be processed as follows:

  For cases defined in +sqlite3, where the fragment identifier resolves
  per the +sqlite3 rules, then as specified in +sqlite3.

  For cases defined in +sqlite3, where the fragment identifier does not
  resolve per the +sqlite3 rules, then as specified in "xxx/yyy+sqlite3".

  For cases not defined in +sqlite3, then as specified in "xxx/yyy+sqlite3".

Security considerations

  Same as for "application/vnd.sqlite3".

  Each individual media type registered with a +sqlite3 suffix can have
  additional security considerations.  For example, if a specific
  registration requires that certain extension functions are available,
  or that blob fields contain data to be processed by other libraries or
  external tools, or if only a single implementation exists to handle
  a specific registered media type, then this increases the known attack
  surface available to an attacker.

Contact

  SQLite mailing list
  <[email-masked]>

Author/Change controller.

  Clemens Ladisch
  <[email-masked]>