README.md
1django-bulk-update
2==================================
3[![Build Status](https://travis-ci.org/aykut/django-bulk-update.svg?branch=master)](https://travis-ci.org/aykut/django-bulk-update)
4[![Coverage Status](https://coveralls.io/repos/aykut/django-bulk-update/badge.svg?branch=master)](https://coveralls.io/r/aykut/django-bulk-update?branch=master)
5
6Simple bulk update over Django ORM or with helper function.
7
8This project aims to bulk update given objects using **one query** over
9**Django ORM**.
10
11Installation
12==================================
13 pip install django-bulk-update
14
15Usage
16==================================
17With manager:
18
19```python
20import random
21from django_bulk_update.manager import BulkUpdateManager
22from tests.models import Person
23
24class Person(models.Model):
25 ...
26 objects = BulkUpdateManager()
27
28random_names = ['Walter', 'The Dude', 'Donny', 'Jesus']
29people = Person.objects.all()
30for person in people:
31 person.name = random.choice(random_names)
32
33Person.objects.bulk_update(people, update_fields=['name']) # updates only name column
34Person.objects.bulk_update(people, exclude_fields=['username']) # updates all columns except username
35Person.objects.bulk_update(people) # updates all columns
36Person.objects.bulk_update(people, batch_size=50000) # updates all columns by 50000 sized chunks
37```
38
39
40With helper:
41
42```python
43import random
44from django_bulk_update.helper import bulk_update
45from tests.models import Person
46
47random_names = ['Walter', 'The Dude', 'Donny', 'Jesus']
48people = Person.objects.all()
49for person in people:
50 person.name = random.choice(random_names)
51
52bulk_update(people, update_fields=['name']) # updates only name column
53bulk_update(people, exclude_fields=['username']) # updates all columns except username
54bulk_update(people, using='someotherdb') # updates all columns using the given db
55bulk_update(people) # updates all columns using the default db
56bulk_update(people, batch_size=50000) # updates all columns by 50000 sized chunks using the default db
57```
58
59Note: You can consider to use `.only('name')` when you only want to update `name`, so that Django will only retrieve name data from db.
60
61And consider to use `.defer('username')` when you don't want to update `username`, so Django won't retrieve username from db.
62These optimization can improve the performance even more.
63
64Performance Tests:
65==================================
66Here we test the performance of the `bulk_update` function vs. simply calling
67`.save()` on every object update (`dmmy_update`). The interesting metric is the speedup using
68the `bulk_update` function more than the actual raw times.
69
70
71```python
72# Note: SQlite is unable to run the `timeit` tests
73# due to the max number of sql variables
74In [1]: import os
75In [2]: import timeit
76In [3]: import django
77
78In [4]: os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.test_settings'
79In [5]: django.setup()
80
81In [6]: from tests.fixtures import create_fixtures
82
83In [7]: django.db.connection.creation.create_test_db()
84In [8]: create_fixtures(1000)
85
86In [9]: setup='''
87import random
88from django_bulk_update import helper
89from tests.models import Person
90random_names = ['Walter', 'The Dude', 'Donny', 'Jesus']
91ids = list(Person.objects.values_list('id', flat=True)[:1000])
92people = Person.objects.filter(id__in=ids)
93for p in people:
94 name = random.choice(random_names)
95 p.name = name
96 p.email = '%s@example.com' % name
97bu_update = lambda: helper.bulk_update(people, update_fields=['name', 'email'])
98'''
99
100In [10]: bu_perf = min(timeit.Timer('bu_update()', setup=setup).repeat(7, 100))
101
102In [11]: setup='''
103import random
104from tests.models import Person
105from django.db.models import F
106random_names = ['Walter', 'The Dude', 'Donny', 'Jesus']
107ids = list(Person.objects.values_list('id', flat=True)[:1000])
108people = Person.objects.filter(id__in=ids)
109def dmmy_update():
110 for p in people:
111 name = random.choice(random_names)
112 p.name = name
113 p.email = '%s@example.com' % name
114 p.save(update_fields=['name', 'email'])
115'''
116
117In [12]: dmmy_perf = min(timeit.Timer('dmmy_update()', setup=setup).repeat(7, 100))
118In [13]: print 'Bulk update performance: %.2f. Dummy update performance: %.2f. Speedup: %.2f.' % (bu_perf, dmmy_perf, dmmy_perf / bu_perf)
119Bulk update performance: 7.05. Dummy update performance: 373.12. Speedup: 52.90.
120```
121
122Requirements
123==================================
124- Django 1.8+
125
126Contributors
127==================================
128- [aykut](https://github.com/aykut)
129- [daleobrien](https://github.com/daleobrien)
130- [sruon](https://github.com/sruon)
131- [HowerHell](https://github.com/HoverHell)
132- [c-nichols](https://github.com/c-nichols)
133- [towr](https://github.com/towr)
134- [joshblum](https://github.com/joshblum)
135- [luzfcb](https://github.com/luzfcb)
136- [torchingloom](https://github.com/torchingloom)
137- [cihann](https://github.com/cihann)
138- [wetneb](https://github.com/wetneb)
139- [tatterdemalion](https://github.com/tatterdemalion)
140- [gabriel-laet](https://github.com/gabriel-laet)
141- [arnau126](https://github.com/arnau126)
142
143TODO
144==================================
145- Geometry Fields support
146
147License
148==================================
149django-bulk-update is released under the MIT License. See the LICENSE file for more details.
150