1.. module:: marshmallow
2
3Extending Schemas
4=================
5
6Pre-processing and Post-processing Methods
7------------------------------------------
8
9Data pre-processing and post-processing methods can be registered using the `pre_load <marshmallow.decorators.pre_load>`, `post_load <marshmallow.decorators.post_load>`, `pre_dump <marshmallow.decorators.pre_dump>`, and `post_dump <marshmallow.decorators.post_dump>` decorators.
10
11
12.. code-block:: python
13
14    from marshmallow import Schema, fields, pre_load
15
16
17    class UserSchema(Schema):
18        name = fields.Str()
19        slug = fields.Str()
20
21        @pre_load
22        def slugify_name(self, in_data, **kwargs):
23            in_data["slug"] = in_data["slug"].lower().strip().replace(" ", "-")
24            return in_data
25
26
27    schema = UserSchema()
28    result = schema.load({"name": "Steve", "slug": "Steve Loria "})
29    result["slug"]  # => 'steve-loria'
30
31Passing "many"
32++++++++++++++
33
34By default, pre- and post-processing methods receive one object/datum at a time, transparently handling the ``many`` parameter passed to the ``Schema``'s :func:`~marshmallow.Schema.dump`/:func:`~marshmallow.Schema.load` method at runtime.
35
36In cases where your pre- and post-processing methods needs to handle the input collection when processing multiple objects, add ``pass_many=True`` to the method decorators.
37
38Your method will then receive the input data (which may be a single datum or a collection, depending on the dump/load call).
39
40Example: Enveloping
41+++++++++++++++++++
42
43One common use case is to wrap data in a namespace upon serialization and unwrap the data during deserialization.
44
45.. code-block:: python
46
47    from marshmallow import Schema, fields, pre_load, post_load, post_dump
48
49
50    class BaseSchema(Schema):
51        # Custom options
52        __envelope__ = {"single": None, "many": None}
53        __model__ = User
54
55        def get_envelope_key(self, many):
56            """Helper to get the envelope key."""
57            key = self.__envelope__["many"] if many else self.__envelope__["single"]
58            assert key is not None, "Envelope key undefined"
59            return key
60
61        @pre_load(pass_many=True)
62        def unwrap_envelope(self, data, many, **kwargs):
63            key = self.get_envelope_key(many)
64            return data[key]
65
66        @post_dump(pass_many=True)
67        def wrap_with_envelope(self, data, many, **kwargs):
68            key = self.get_envelope_key(many)
69            return {key: data}
70
71        @post_load
72        def make_object(self, data, **kwargs):
73            return self.__model__(**data)
74
75
76    class UserSchema(BaseSchema):
77        __envelope__ = {"single": "user", "many": "users"}
78        __model__ = User
79        name = fields.Str()
80        email = fields.Email()
81
82
83    user_schema = UserSchema()
84
85    user = User("Mick", email="mick@stones.org")
86    user_data = user_schema.dump(user)
87    # {'user': {'email': 'mick@stones.org', 'name': 'Mick'}}
88
89    users = [
90        User("Keith", email="keith@stones.org"),
91        User("Charlie", email="charlie@stones.org"),
92    ]
93    users_data = user_schema.dump(users, many=True)
94    # {'users': [{'email': 'keith@stones.org', 'name': 'Keith'},
95    #            {'email': 'charlie@stones.org', 'name': 'Charlie'}]}
96
97    user_objs = user_schema.load(users_data, many=True)
98    # [<User(name='Keith Richards')>, <User(name='Charlie Watts')>]
99
100
101Raising Errors in Pre-/Post-processor Methods
102+++++++++++++++++++++++++++++++++++++++++++++
103
104Pre- and post-processing methods may raise a `ValidationError <marshmallow.exceptions.ValidationError>`. By default, errors will be stored on the ``"_schema"`` key in the errors dictionary.
105
106.. code-block:: python
107
108    from marshmallow import Schema, fields, ValidationError, pre_load
109
110
111    class BandSchema(Schema):
112        name = fields.Str()
113
114        @pre_load
115        def unwrap_envelope(self, data, **kwargs):
116            if "data" not in data:
117                raise ValidationError('Input data must have a "data" key.')
118            return data["data"]
119
120
121    sch = BandSchema()
122    try:
123        sch.load({"name": "The Band"})
124    except ValidationError as err:
125        err.messages
126    # {'_schema': ['Input data must have a "data" key.']}
127
128If you want to store and error on a different key, pass the key name as the second argument to `ValidationError <marshmallow.exceptions.ValidationError>`.
129
130.. code-block:: python
131
132    from marshmallow import Schema, fields, ValidationError, pre_load
133
134
135    class BandSchema(Schema):
136        name = fields.Str()
137
138        @pre_load
139        def unwrap_envelope(self, data, **kwargs):
140            if "data" not in data:
141                raise ValidationError(
142                    'Input data must have a "data" key.', "_preprocessing"
143                )
144            return data["data"]
145
146
147    sch = BandSchema()
148    try:
149        sch.load({"name": "The Band"})
150    except ValidationError as err:
151        err.messages
152    # {'_preprocessing': ['Input data must have a "data" key.']}
153
154
155Pre-/Post-processor Invocation Order
156++++++++++++++++++++++++++++++++++++
157
158In summary, the processing pipeline for deserialization is as follows:
159
1601. ``@pre_load(pass_many=True)`` methods
1612. ``@pre_load(pass_many=False)`` methods
1623. ``load(in_data, many)`` (validation and deserialization)
1634. ``@validates`` methods (field validators)
1645. ``@validates_schema`` methods (schema validators)
1656. ``@post_load(pass_many=True)`` methods
1667. ``@post_load(pass_many=False)`` methods
167
168The pipeline for serialization is similar, except that the ``pass_many=True`` processors are invoked *after* the ``pass_many=False`` processors and there are no validators.
169
1701. ``@pre_dump(pass_many=False)`` methods
1712. ``@pre_dump(pass_many=True)`` methods
1723. ``dump(obj, many)`` (serialization)
1734. ``@post_dump(pass_many=False)`` methods
1745. ``@post_dump(pass_many=True)`` methods
175
176
177.. warning::
178
179    You may register multiple processor methods on a Schema. Keep in mind, however, that **the invocation order of decorated methods of the same type is not guaranteed**. If you need to guarantee order of processing steps, you should put them in the same method.
180
181
182    .. code-block:: python
183
184        from marshmallow import Schema, fields, pre_load
185
186        # YES
187        class MySchema(Schema):
188            field_a = fields.Field()
189
190            @pre_load
191            def preprocess(self, data, **kwargs):
192                step1_data = self.step1(data)
193                step2_data = self.step2(step1_data)
194                return step2_data
195
196            def step1(self, data):
197                do_step1(data)
198
199            # Depends on step1
200            def step2(self, data):
201                do_step2(data)
202
203
204        # NO
205        class MySchema(Schema):
206            field_a = fields.Field()
207
208            @pre_load
209            def step1(self, data, **kwargs):
210                do_step1(data)
211
212            # Depends on step1
213            @pre_load
214            def step2(self, data, **kwargs):
215                do_step2(data)
216
217
218.. _schemavalidation:
219
220Schema-level Validation
221-----------------------
222
223You can register schema-level validation functions for a :class:`Schema` using the `marshmallow.validates_schema <marshmallow.decorators.validates_schema>` decorator. By default, schema-level validation errors will be stored on the ``_schema`` key of the errors dictionary.
224
225.. code-block:: python
226
227    from marshmallow import Schema, fields, validates_schema, ValidationError
228
229
230    class NumberSchema(Schema):
231        field_a = fields.Integer()
232        field_b = fields.Integer()
233
234        @validates_schema
235        def validate_numbers(self, data, **kwargs):
236            if data["field_b"] >= data["field_a"]:
237                raise ValidationError("field_a must be greater than field_b")
238
239
240    schema = NumberSchema()
241    try:
242        schema.load({"field_a": 1, "field_b": 2})
243    except ValidationError as err:
244        err.messages["_schema"]
245    # => ["field_a must be greater than field_b"]
246
247Storing Errors on Specific Fields
248+++++++++++++++++++++++++++++++++
249
250It is possible to report errors on fields and subfields using a `dict`.
251
252When multiple schema-leval validator return errors, the error structures are merged together in the :exc:`ValidationError <marshmallow.exceptions.ValidationError>` raised at the end of the validation.
253
254.. code-block:: python
255
256    from marshmallow import Schema, fields, validates_schema, ValidationError
257
258
259    class NumberSchema(Schema):
260        field_a = fields.Integer()
261        field_b = fields.Integer()
262        field_c = fields.Integer()
263        field_d = fields.Integer()
264
265        @validates_schema
266        def validate_lower_bound(self, data, **kwargs):
267            errors = {}
268            if data["field_b"] <= data["field_a"]:
269                errors["field_b"] = ["field_b must be greater than field_a"]
270            if data["field_c"] <= data["field_a"]:
271                errors["field_c"] = ["field_c must be greater than field_a"]
272            if errors:
273                raise ValidationError(errors)
274
275        @validates_schema
276        def validate_upper_bound(self, data, **kwargs):
277            errors = {}
278            if data["field_b"] >= data["field_d"]:
279                errors["field_b"] = ["field_b must be lower than field_d"]
280            if data["field_c"] >= data["field_d"]:
281                errors["field_c"] = ["field_c must be lower than field_d"]
282            if errors:
283                raise ValidationError(errors)
284
285
286    schema = NumberSchema()
287    try:
288        schema.load({"field_a": 3, "field_b": 2, "field_c": 1, "field_d": 0})
289    except ValidationError as err:
290        err.messages
291    # => {
292    #     'field_b': [
293    #         'field_b must be greater than field_a',
294    #         'field_b must be lower than field_d'
295    #     ],
296    #     'field_c': [
297    #         'field_c must be greater than field_a',
298    #         'field_c must be lower than field_d'
299    #     ]
300    #    }
301
302
303Using Original Input Data
304-------------------------
305
306If you want to use the original, unprocessed input, you can add ``pass_original=True`` to
307`post_load <marshmallow.decorators.post_load>` or `validates_schema <marshmallow.decorators.validates_schema>`.
308
309.. code-block:: python
310
311    from marshmallow import Schema, fields, post_load, ValidationError
312
313
314    class MySchema(Schema):
315        foo = fields.Int()
316        bar = fields.Int()
317
318        @post_load(pass_original=True)
319        def add_baz_to_bar(self, data, original_data, **kwargs):
320            baz = original_data.get("baz")
321            if baz:
322                data["bar"] = data["bar"] + baz
323            return data
324
325
326    schema = MySchema()
327    schema.load({"foo": 1, "bar": 2, "baz": 3})
328    # {'foo': 1, 'bar': 5}
329
330.. seealso::
331
332   The default behavior for unspecified fields can be controlled with the ``unknown`` option, see :ref:`Handling Unknown Fields <unknown>` for more information.
333
334Overriding How Attributes Are Accessed
335--------------------------------------
336
337By default, marshmallow uses `utils.get_value` to pull attributes from various types of objects for serialization. This will work for *most* use cases.
338
339However, if you want to specify how values are accessed from an object, you can override the :meth:`get_attribute <marshmallow.Schema.get_attribute>` method.
340
341.. code-block:: python
342
343    class UserDictSchema(Schema):
344        name = fields.Str()
345        email = fields.Email()
346
347        # If we know we're only serializing dictionaries, we can
348        # use dict.get for all input objects
349        def get_attribute(self, obj, key, default):
350            return obj.get(key, default)
351
352Custom Error Handling
353---------------------
354
355By default, :meth:`Schema.load` will raise a :exc:`ValidationError <marshmallow.exceptions.ValidationError>` if passed invalid data.
356
357You can specify a custom error-handling function for a :class:`Schema` by overriding the `handle_error <marshmallow.Schema.handle_error>`  method. The method receives the :exc:`ValidationError <marshmallow.exceptions.ValidationError>` and the original input data to be deserialized.
358
359.. code-block:: python
360
361    import logging
362    from marshmallow import Schema, fields
363
364
365    class AppError(Exception):
366        pass
367
368
369    class UserSchema(Schema):
370        email = fields.Email()
371
372        def handle_error(self, exc, data, **kwargs):
373            """Log and raise our custom exception when (de)serialization fails."""
374            logging.error(exc.messages)
375            raise AppError("An error occurred with input: {0}".format(data))
376
377
378    schema = UserSchema()
379    schema.load({"email": "invalid-email"})  # raises AppError
380
381
382Custom "class Meta" Options
383---------------------------
384
385``class Meta`` options are a way to configure and modify a :class:`Schema's <Schema>` behavior. See the :class:`API docs <Schema.Meta>` for a listing of available options.
386
387You can add custom ``class Meta`` options by subclassing :class:`SchemaOpts`.
388
389Example: Enveloping, Revisited
390++++++++++++++++++++++++++++++
391
392Let's build upon the example above for adding an envelope to serialized output. This time, we will allow the envelope key to be customizable with ``class Meta`` options.
393
394::
395
396    # Example outputs
397    {
398        'user': {
399            'name': 'Keith',
400            'email': 'keith@stones.com'
401        }
402    }
403    # List output
404    {
405        'users': [{'name': 'Keith'}, {'name': 'Mick'}]
406    }
407
408
409First, we'll add our namespace configuration to a custom options class.
410
411.. code-block:: python
412
413    from marshmallow import Schema, SchemaOpts
414
415
416    class NamespaceOpts(SchemaOpts):
417        """Same as the default class Meta options, but adds "name" and
418        "plural_name" options for enveloping.
419        """
420
421        def __init__(self, meta, **kwargs):
422            SchemaOpts.__init__(self, meta, **kwargs)
423            self.name = getattr(meta, "name", None)
424            self.plural_name = getattr(meta, "plural_name", self.name)
425
426
427Then we create a custom :class:`Schema` that uses our options class.
428
429.. code-block:: python
430
431    class NamespacedSchema(Schema):
432        OPTIONS_CLASS = NamespaceOpts
433
434        @pre_load(pass_many=True)
435        def unwrap_envelope(self, data, many, **kwargs):
436            key = self.opts.plural_name if many else self.opts.name
437            return data[key]
438
439        @post_dump(pass_many=True)
440        def wrap_with_envelope(self, data, many, **kwargs):
441            key = self.opts.plural_name if many else self.opts.name
442            return {key: data}
443
444
445Our application schemas can now inherit from our custom schema class.
446
447.. code-block:: python
448
449    class UserSchema(NamespacedSchema):
450        name = fields.String()
451        email = fields.Email()
452
453        class Meta:
454            name = "user"
455            plural_name = "users"
456
457
458    ser = UserSchema()
459    user = User("Keith", email="keith@stones.com")
460    result = ser.dump(user)
461    result  # {"user": {"name": "Keith", "email": "keith@stones.com"}}
462
463Using Context
464-------------
465
466The ``context`` attribute of a `Schema` is a general-purpose store for extra information that may be needed for (de)serialization. It may be used in both ``Schema`` and ``Field`` methods.
467
468.. code-block:: python
469
470    schema = UserSchema()
471    # Make current HTTP request available to
472    # custom fields, schema methods, schema validators, etc.
473    schema.context["request"] = request
474    schema.dump(user)
475
476Custom Error Messages
477---------------------
478
479To customize the schema-level error messages that `load <marshmallow.Schema.load>` and `loads <marshmallow.Schema.loads>` use when raising a `ValidationError <marshmallow.exceptions.ValidationError>`, override the `error_messages <marshmallow.Schema.error_messages>` class variable:
480
481.. code-block:: python
482
483    class MySchema(Schema):
484        error_messages = {
485            "unknown": "Custom unknown field error message.",
486            "type": "Custom invalid type error message.",
487        }
488
489
490Field-level error message defaults can be set on `Field.default_error_messages <marshmallow.fields.Field.default_error_messages>`.
491
492
493.. code-block:: python
494
495   from marshmallow import Schema, fields
496
497   fields.Field.default_error_messages["required"] = "You missed something!"
498
499
500   class ArtistSchema(Schema):
501       name = fields.Str(required=True)
502       label = fields.Str(required=True, error_messages={"required": "Label missing."})
503
504
505   print(ArtistSchema().validate({}))
506   # {'label': ['Label missing.'], 'name': ['You missed something!']}
507