• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..29-Jan-2021-

README.mdH A D29-Jan-20214.7 KiB11399

arena.goH A D29-Jan-20213.9 KiB13776

skl.goH A D29-Jan-202114.5 KiB517320

README.md

1This is much better than `skiplist` and `slist`.
2
3```
4BenchmarkReadWrite/frac_0-8         	 3000000	       537 ns/op
5BenchmarkReadWrite/frac_1-8         	 3000000	       503 ns/op
6BenchmarkReadWrite/frac_2-8         	 3000000	       492 ns/op
7BenchmarkReadWrite/frac_3-8         	 3000000	       475 ns/op
8BenchmarkReadWrite/frac_4-8         	 3000000	       440 ns/op
9BenchmarkReadWrite/frac_5-8         	 5000000	       442 ns/op
10BenchmarkReadWrite/frac_6-8         	 5000000	       380 ns/op
11BenchmarkReadWrite/frac_7-8         	 5000000	       338 ns/op
12BenchmarkReadWrite/frac_8-8         	 5000000	       294 ns/op
13BenchmarkReadWrite/frac_9-8         	10000000	       268 ns/op
14BenchmarkReadWrite/frac_10-8        	100000000	        26.3 ns/op
15```
16
17And even better than a simple map with read-write lock:
18
19```
20BenchmarkReadWriteMap/frac_0-8         	 2000000	       774 ns/op
21BenchmarkReadWriteMap/frac_1-8         	 2000000	       647 ns/op
22BenchmarkReadWriteMap/frac_2-8         	 3000000	       605 ns/op
23BenchmarkReadWriteMap/frac_3-8         	 3000000	       603 ns/op
24BenchmarkReadWriteMap/frac_4-8         	 3000000	       556 ns/op
25BenchmarkReadWriteMap/frac_5-8         	 3000000	       472 ns/op
26BenchmarkReadWriteMap/frac_6-8         	 3000000	       476 ns/op
27BenchmarkReadWriteMap/frac_7-8         	 3000000	       457 ns/op
28BenchmarkReadWriteMap/frac_8-8         	 5000000	       444 ns/op
29BenchmarkReadWriteMap/frac_9-8         	 5000000	       361 ns/op
30BenchmarkReadWriteMap/frac_10-8        	10000000	       212 ns/op
31```
32
33# Node Pooling
34
35Command used
36
37```
38rm -Rf tmp && /usr/bin/time -l ./populate -keys_mil 10
39```
40
41For pprof results, we run without using /usr/bin/time. There are four runs below.
42
43Results seem to vary quite a bit between runs.
44
45## Before node pooling
46
47```
481311.53MB of 1338.69MB total (97.97%)
49Dropped 30 nodes (cum <= 6.69MB)
50Showing top 10 nodes out of 37 (cum >= 12.50MB)
51      flat  flat%   sum%        cum   cum%
52  523.04MB 39.07% 39.07%   523.04MB 39.07%  github.com/dgraph-io/badger/skl.(*Skiplist).Put
53  184.51MB 13.78% 52.85%   184.51MB 13.78%  runtime.stringtoslicebyte
54  166.01MB 12.40% 65.25%   689.04MB 51.47%  github.com/dgraph-io/badger/mem.(*Table).Put
55     165MB 12.33% 77.58%      165MB 12.33%  runtime.convT2E
56  116.92MB  8.73% 86.31%   116.92MB  8.73%  bytes.makeSlice
57   62.50MB  4.67% 90.98%    62.50MB  4.67%  main.newValue
58   34.50MB  2.58% 93.56%    34.50MB  2.58%  github.com/dgraph-io/badger/table.(*BlockIterator).parseKV
59   25.50MB  1.90% 95.46%   100.06MB  7.47%  github.com/dgraph-io/badger/y.(*MergeIterator).Next
60   21.06MB  1.57% 97.04%    21.06MB  1.57%  github.com/dgraph-io/badger/table.(*Table).read
61   12.50MB  0.93% 97.97%    12.50MB  0.93%  github.com/dgraph-io/badger/table.header.Encode
62
63      128.31 real       329.37 user        17.11 sys
643355660288  maximum resident set size
65         0  average shared memory size
66         0  average unshared data size
67         0  average unshared stack size
68   2203080  page reclaims
69       764  page faults
70         0  swaps
71       275  block input operations
72        76  block output operations
73         0  messages sent
74         0  messages received
75         0  signals received
76     49173  voluntary context switches
77    599922  involuntary context switches
78```
79
80## After node pooling
81
82```
831963.13MB of 2026.09MB total (96.89%)
84Dropped 29 nodes (cum <= 10.13MB)
85Showing top 10 nodes out of 41 (cum >= 185.62MB)
86      flat  flat%   sum%        cum   cum%
87  658.05MB 32.48% 32.48%   658.05MB 32.48%  github.com/dgraph-io/badger/skl.glob..func1
88  297.51MB 14.68% 47.16%   297.51MB 14.68%  runtime.convT2E
89  257.51MB 12.71% 59.87%   257.51MB 12.71%  runtime.stringtoslicebyte
90  249.01MB 12.29% 72.16%  1007.06MB 49.70%  github.com/dgraph-io/badger/mem.(*Table).Put
91  142.43MB  7.03% 79.19%   142.43MB  7.03%  bytes.makeSlice
92     100MB  4.94% 84.13%   758.05MB 37.41%  github.com/dgraph-io/badger/skl.newNode
93   99.50MB  4.91% 89.04%    99.50MB  4.91%  main.newValue
94      75MB  3.70% 92.74%       75MB  3.70%  github.com/dgraph-io/badger/table.(*BlockIterator).parseKV
95   44.62MB  2.20% 94.94%    44.62MB  2.20%  github.com/dgraph-io/badger/table.(*Table).read
96   39.50MB  1.95% 96.89%   185.62MB  9.16%  github.com/dgraph-io/badger/y.(*MergeIterator).Next
97
98      135.58 real       374.29 user        17.65 sys
993740614656  maximum resident set size
100         0  average shared memory size
101         0  average unshared data size
102         0  average unshared stack size
103   2276566  page reclaims
104       770  page faults
105         0  swaps
106       128  block input operations
107        90  block output operations
108         0  messages sent
109         0  messages received
110         0  signals received
111     46434  voluntary context switches
112    597049  involuntary context switches
113```