1// Copyright 2019 The Go Cloud Development Kit Authors 2// 3// Licensed under the Apache License, Version 2.0 (the "License"); 4// you may not use this file except in compliance with the License. 5// You may obtain a copy of the License at 6// 7// https://www.apache.org/licenses/LICENSE-2.0 8// 9// Unless required by applicable law or agreed to in writing, software 10// distributed under the License is distributed on an "AS IS" BASIS, 11// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12// See the License for the specific language governing permissions and 13// limitations under the License. 14 15// Package docstore provides a portable way of interacting with a document store. 16// Subpackages contain driver implementations of docstore for supported 17// services. 18// 19// See https://gocloud.dev/howto/docstore/ for a detailed how-to guide. 20// 21// 22// Collections 23// 24// In docstore, documents are grouped into collections, and each document has a key 25// that is unique in its collection. You can add, retrieve, modify and delete 26// documents by key, and you can query a collection to retrieve documents that match 27// certain criteria. 28// 29// 30// Representing Documents 31// 32// A document is a set of named fields, each with a value. A field's value can be a scalar, 33// a list, or a nested document. 34// 35// Docstore allows you to represent documents as either map[string]interface{} or 36// struct pointers. When you represent a document as a map, the fields are map keys 37// and the values are map values. Lists are represented with slices. For example, 38// here is a document about a book described as a map: 39// 40// doc := map[string]interface{}{ 41// "Title": "The Master and Margarita", 42// "Author": map[string]interface{}{ 43// "First": "Mikhail", 44// "Last": "Bulgakov", 45// }, 46// "PublicationYears": []int{1967, 1973}, 47// } 48// 49// Note that the value of "PublicationYears" is a list, and the value of "Author" is 50// itself a document. 51// 52// Here is the same document represented with structs: 53// 54// type Book struct { 55// Title string 56// Author Name 57// PublicationYears []int 58// } 59// 60// type Name struct { 61// First, Last string 62// } 63// 64// doc := &Book{ 65// Title: "The Master and Margarita", 66// Author: Name{ 67// First: "Mikhail", 68// Last: "Bulgakov", 69// }, 70// PublicationYears: []int{1967, 1973}, 71// } 72// 73// You must use a pointer to a struct to represent a document, although structs 74// nested inside a document, like the Name struct above, need not be pointers. 75// 76// Maps are best for applications where you don't know the structure of the 77// documents. Using structs is preferred because it enforces some structure on your 78// data. 79// 80// By default, Docstore treats a struct's exported fields as the fields of the 81// document. You can alter this default mapping by using a struct tag beginning 82// with "docstore:". Docstore struct tags support renaming, omitting fields 83// unconditionally, or omitting them only when they are empty, exactly like 84// encoding/json. For example, this is the Book struct with different field 85// names: 86// 87// type Book struct { 88// Title string `docstore:"title"` 89// Author Name `docstore:"author"` 90// PublicationYears []int `docstore:"pub_years,omitempty"` 91// NumPublications int `docstore:"-"` 92// } 93// 94// This struct describes a document with field names "title", "author" and 95// "pub_years". The pub_years field is omitted from the stored document if it has 96// length zero. The NumPublications field is never stored because it can easily be 97// computed from the PublicationYears field. 98// 99// Given a document field "Foo" and a struct type document, Docstore's decoder 100// will look through the destination struct's field to find (in order of 101// preference): 102// - An exported field with a tag of "Foo"; 103// - An exported field named "Foo". 104// 105// Note that unlike encoding/json, Docstore does case-sensitive matching during 106// decoding to match the behavior of decoders in most docstore services. 107// 108// 109// Representing Data 110// 111// Values stored in document fields can be any of a wide range of types. All 112// primitive types except for complex numbers are supported, as well as slices and 113// maps (the map key type must be a string, an integer, or a type that implements 114// encoding.TextMarshaler). In addition, any type that implements 115// encoding.BinaryMarshaler or encoding.TextMarshaler is permitted. This set of types 116// closely matches the encoding/json package (see https://golang.org/pkg/encoding/json). 117// 118// Times deserve special mention. Docstore can store and retrieve values of type 119// time.Time, with two caveats. First, the timezone will not be preserved. Second, 120// Docstore guarantees only that time.Time values are represented to millisecond 121// precision. Many services will do better, but if you need to be sure that times 122// are stored with nanosecond precision, convert the time.Time to another type before 123// storing and re-create when you retrieve it. For instance, if you store Unix 124// time in nanoseconds using time's UnixNano method, you can get the original 125// time back (in the local timezone) with the time.Unix function. 126// 127// 128// Representing Keys 129// 130// The key of a docstore document is its unique identifier, usually a field. 131// Keys never appear alone in the docstore API, only as part of a document. For 132// instance, to retrieve a document by key, you pass the Collection.Get method 133// a document as a struct pointer or map with the key field populated, and docstore 134// populates the rest of that argument with the stored contents. Docstore 135// doesn't take zero-value key. 136// 137// When you open a collection using an OpenCollection method of the 138// service-specific driver or a URL, you specify how to extract the key from a 139// document. 140// Usually, you provide the name of the key field, as in the example below: 141// 142// coll, err := memdocstore.OpenCollection("SSN", nil) 143// 144// Here, the "SSN" field of the document is used as the key. Some drivers let you 145// supply a function to extract the key from the document, which can be useful if the 146// key is composed of more than one field. 147// 148// 149// Actions 150// 151// Docstore supports six actions on documents as methods on the Collection type: 152// - Get retrieves a document. 153// - Create creates a new document. 154// - Replace replaces an existing document. 155// - Put puts a document into a collection, replacing it if it is already present. 156// - Update applies a set of modifications to a document. 157// - Delete deletes a document. 158// 159// Each action acts atomically on a single document. You can execute actions 160// individually or you can group them into an action list, like so: 161// 162// err := coll.Actions().Put(doc1).Replace(doc2).Get(doc3).Do(ctx) 163// 164// When you use an action list, docstore will try to optimize the execution of the 165// actions. For example, multiple Get actions may be combined into a single "batch 166// get" RPC. For the most part, actions in a list execute in an undefined order 167// (perhaps concurrently) and independently, but read and write operations on the same 168// document are executed in the user-specified order. See the documentation of 169// ActionList for details. 170// 171// 172// Revisions 173// 174// Docstore supports document revisions to distinguish different versions of a 175// document and enable optimistic locking. By default, Docstore stores the 176// revision in the field named "DocstoreRevision" (stored in the constant 177// DefaultRevisionField). Providers give you the option of changing that field 178// name. 179// 180// When you pass a document with a revision field to a write action, Docstore 181// will give it a revision at creation time or update the revision value when 182// modifying the document. If you don't want Docstore to handle any revision 183// logic, simply do not have the revision field in your document. 184// 185// When you pass a document with a non-nil revision to Put, Replace, Update or 186// Delete, Docstore will also compare the revision of the stored document to 187// that of the given document before making the change. It returns an error with 188// code FailedPrecondition on mismatch. (See https://gocloud.dev/gcerrors for 189// information about error codes.) If modification methods are called on a 190// document struct or map a nil revision field, then no revision checks are 191// performed, and changes are forced blindly, but a new revision will still be 192// given for the document. For example, if you call Get to retrieve a document 193// with a revision, then later perform a write action with that same document, 194// it will fail if the document was changed since the Get. 195// 196// Since different services use different types for revisions, revision fields 197// of unspecified type must be handled. When defining a document struct, 198// define the field to be of type interface{}. For example, 199// 200// type User { 201// Name string 202// DocstoreRevision interface{} 203// } 204// 205// 206// Queries 207// 208// Docstore supports querying within a collection. Call the Query method on 209// Collection to obtain a Query value, then build your query by calling Query methods 210// like Where, Limit and so on. Finally, call the Get method on the query to execute it. 211// The result is an iterator, whose use is described below. 212// 213// iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx) 214// 215// The Where method defines a filter condition, much like a WHERE clause in SQL. 216// Conditions are of the form "field op value", where field is any document field 217// path (including dot-separated paths), op is one of "=", ">", "<", ">=" or "<=", 218// and value can be any value. 219// 220// iter := coll.Query().Where("Author.Last", "=", "Bulgakov").Limit(3).Get(ctx) 221// 222// You can make multiple Where calls. In some cases, parts of a Where clause may be 223// processed in the driver rather than natively by the backing service, which may have 224// performance implications for large result sets. See the driver package 225// documentation for details. 226// 227// Use the DocumentIterator returned from Query.Get by repeatedly calling its Next 228// method until it returns io.EOF. Always call Stop when you are finished with an 229// iterator. It is wise to use a defer statement for this. 230// 231// iter := coll.Query().Where("size", ">", 10).Limit(5).Get(ctx) 232// defer iter.Stop() 233// for { 234// m := map[string]interface{}{} 235// err := iter.Next(ctx, m) 236// if err == io.EOF { 237// break 238// } 239// if err != nil { 240// return err 241// } 242// fmt.Println(m) 243// } 244// 245// 246// Errors 247// 248// The errors returned from this package can be inspected in several ways: 249// 250// The Code function from https://gocloud.dev/gcerrors will return an error code, also 251// defined in that package, when invoked on an error. 252// 253// The Collection.ErrorAs method can retrieve the underlying driver error from 254// the returned error. See the specific driver's package doc for the supported 255// types. 256// 257// 258// OpenCensus Integration 259// 260// OpenCensus supports tracing and metric collection for multiple languages and 261// backend providers. See https://opencensus.io. 262// 263// This API collects OpenCensus traces and metrics for the following methods: 264// - ActionList.Do 265// - Query.Get (for the first query only; drivers may make additional calls while iterating over results) 266// All trace and metric names begin with the package import path. 267// The traces add the method name. 268// For example, "gocloud.dev/docstore/ActionList.Do". 269// The metrics are "completed_calls", a count of completed method calls by driver, 270// method and status (error code); and "latency", a distribution of method latency 271// by driver and method. 272// For example, "gocloud.dev/docstore/latency". 273// 274// To enable trace collection in your application, see "Configure Exporter" at 275// https://opencensus.io/quickstart/go/tracing. 276// To enable metric collection in your application, see "Exporting stats" at 277// https://opencensus.io/quickstart/go/metrics. 278package docstore // import "gocloud.dev/docstore" 279