README.md
1# segyio #
2
3[![Travis](https://img.shields.io/travis/equinor/segyio/master.svg?label=travis)](https://travis-ci.org/equinor/segyio)
4[![Appveyor](https://ci.appveyor.com/api/projects/status/2i5cr8ui2t9qbxk9?svg=true)](https://ci.appveyor.com/project/statoil-travis/segyio)
5[![PyPI Updates](https://pyup.io/repos/github/equinor/segyio/shield.svg)](https://pyup.io/repos/github/equinor/segyio/)
6[![Python 3](https://pyup.io/repos/github/equinor/segyio/python-3-shield.svg)](https://pyup.io/repos/github/equinor/segyio/)
7
8## Documentation ##
9
10The official documentation is hosted on [readthedocs](https://segyio.readthedocs.io/).
11
12## Index ##
13
14* [Introduction](#introduction)
15* [Feature summary](#feature-summary)
16* [Getting started](#getting-started)
17 * [Quick start](#quick-start)
18 * [Get segyio](#get-segyio)
19 * [Build segyio](#build-segyio)
20* [Tutorial](#tutorial)
21 * [Basics](#basics)
22 * [Modes](#modes)
23 * [Mode examples](#mode-examples)
24* [Goals](#project-goals)
25* [Contributing](#contributing)
26* [Examples](#examples)
27* [Common issues](#common-issues)
28* [History](#history)
29
30## Introduction ##
31
32Segyio is a small LGPL licensed C library for easy interaction with SEG-Y and
33Seismic Unix formatted seismic data, with language bindings for Python and
34Matlab. Segyio is an attempt to create an easy-to-use, embeddable,
35community-oriented library for seismic applications. Features are added as they
36are needed; suggestions and contributions of all kinds are very welcome.
37
38To catch up on the latest development and features, see the
39[changelog](changelog.md). To write future proof code, consult the planned
40[breaking changes](breaking-changes.md).
41
42## Feature summary ##
43
44 * A low-level C interface with few assumptions; easy to bind to other
45 languages
46 * Read and write binary and textual headers
47 * Read and write traces and trace headers
48 * Simple, powerful, and native-feeling Python interface with numpy
49 integration
50 * Read and write seismic unix files
51 * xarray integration with netcdf_segy
52 * Some simple applications with unix philosophy
53
54## Getting started ##
55
56When segyio is built and installed, you're ready to start programming! Check
57out the [tutorial](#tutorial), [examples](#examples), [example
58programs](python/examples), and [example
59notebooks](https://github.com/equinor/segyio-notebooks). For a technical
60reference with examples and small recipes, [read the
61docs](https://segyio.readthedocs.io/). API docs are also available with pydoc -
62start your favourite Python interpreter and type `help(segyio)`, which should
63integrate well with IDLE, pycharm and other Python tools.
64
65### Quick start ###
66```python
67import segyio
68import numpy as np
69with segyio.open('file.sgy') as f:
70 for trace in f.trace:
71 filtered = trace[np.where(trace < 1e-2)]
72```
73
74See the [examples](#examples) for more.
75
76### Get segyio ###
77
78A copy of segyio is available both as pre-built binaries and source code:
79
80* In Debian [unstable](https://packages.debian.org/source/sid/segyio)
81 * `apt install python3-segyio`
82* Wheels for Python from [PyPI](https://pypi.python.org/pypi/segyio/)
83 * `pip install segyio`
84* Source code from [github](https://github.com/equinor/segyio)
85 * `git clone https://github.com/statoil/segyio`
86* Source code in [tarballs](https://github.com/equinor/segyio/releases)
87
88### Build segyio ###
89
90To build segyio you need:
91 * A C99 compatible C compiler (tested mostly on gcc and clang)
92 * A C++ compiler for the Python extension, and C++11 for the tests
93 * [CMake](https://cmake.org/) version 2.8.12 or greater
94 * [Python](https://www.python.org/) 3.6 or greater
95 * [numpy](http://www.numpy.org/) version 1.10 or greater
96 * [setuptools](https://pypi.python.org/pypi/setuptools) version 28 or greater
97 * [setuptools-scm](https://pypi.python.org/pypi/setuptools_scm)
98 * [pytest](https://pypi.org/project/pytest)
99
100 To build the documentation, you also need
101 [sphinx](https://pypi.org/project/Sphinx)
102
103To build and install segyio, perform the following actions in your console:
104
105```bash
106git clone https://github.com/equinor/segyio
107mkdir segyio/build
108cd segyio/build
109cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
110make
111make install
112```
113
114`make install` must be done as root for a system install; if you want to
115install in your home directory, add `-DCMAKE_INSTALL_PREFIX=~/` or some other
116appropriate directory, or `make DESTDIR=~/ install`. Please ensure your
117environment picks up on non-standard install locations (PYTHONPATH,
118LD_LIBRARY_PATH and PATH).
119
120If you have multiple Python installations, or want to use some alternative
121interpreter, you can help cmake find the right one by passing
122`-DPYTHON_EXECUTABLE=/opt/python/binary` along with install prefix and build
123type.
124
125To build the matlab bindings, invoke CMake with the option `-DBUILD_MEX=ON`. In
126some environments the Matlab binaries are in a non-standard location, in which
127case you need to help CMake find the matlab binaries by passing
128`-DMATLAB_ROOT=/path/to/matlab`.
129
130#### Developers ####
131
132It's recommended to build in debug mode to get more warnings and to embed debug
133symbols in the objects. Substituting `Debug` for `Release` in the
134`CMAKE_BUILD_TYPE` is plenty.
135
136Tests are located in the language/tests directories, and it's highly
137recommended that new features added are demonstrated for correctness and
138contract by adding a test. All tests can be run by invoking `ctest`. Feel free
139to use the tests already written as a guide.
140
141After building segyio you can run the tests with `ctest`, executed from the
142build directory.
143
144Please note that to run the Python examples you need to let your environment
145know where to find the Python library. It can be installed as a user, or on
146adding the segyio/build/python library to your pythonpath.
147
148## Tutorial ##
149
150All code in this tutorial assumes segyio is imported, and that numpy is
151available as np.
152
153```python
154import segyio
155import numpy as np
156```
157
158This tutorial assumes you're familiar with Python and numpy. For a refresh,
159check out the [python tutorial](https://docs.python.org/3/tutorial/) and [numpy
160quickstart](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html)
161
162### Basics ###
163
164Opening a file for reading is done with the `segyio.open` function, and
165idiomatically used with context managers. Using the `with` statement, files are
166properly closed even in the case of exceptions. By default, files are opened
167read-only.
168
169```python
170with segyio.open(filename) as f:
171 ...
172```
173
174Open accepts several options (for more a more comprehensive reference, check
175the open function's docstring with `help(segyio.open)`. The most important
176option is the second (optional) positional argument. To open a file for
177writing, do `segyio.open(filename, 'r+')`, from the C `fopen` function.
178
179Files can be opened in *unstructured* mode, either by passing `segyio.open` the
180optional arguments `strict=False`, in which case not establishing structure
181(inline numbers, crossline numbers etc.) is not an error, and
182`ignore_geometry=True`, in which case segyio won't even try to set these
183internal attributes.
184
185The segy file object has several public attributes describing this structure:
186* `f.ilines`
187 Inferred inline numbers
188* `f.xlines`
189 Inferred crossline numbers
190* `f.offsets`
191 Inferred offsets numbers
192* `f.samples`
193 Inferred sample offsets (frequency and recording time delay)
194* `f.unstructured`
195 True if unstructured, False if structured
196* `f.ext_headers`
197 The number of extended textual headers
198
199If the file is opened *unstructured*, all the line properties will will be
200`None`.
201
202### Modes ###
203
204In segyio, data is retrieved and written through so-called *modes*. Modes are
205abstract arrays, or addressing schemes, and change what names and indices mean.
206All modes are properties on the file handle object, support the `len` function,
207and reads and writes are done through `f.mode[]`. Writes are done with
208assignment. Modes support array slicing inspired by numpy. The following modes
209are available:
210
211* `trace`
212
213 The trace mode offers raw addressing of traces as they are laid out in the
214 file. This, along with `header`, is the only mode available for
215 unstructured files. Traces are enumerated `0..len(f.trace)`.
216
217 Reading a trace yields a numpy `ndarray`, and reading multiple traces
218 yields a generator of `ndarray`s. Generator semantics are used and the same
219 object is reused, so if you want to cache or address trace data later, you
220 must explicitly copy.
221
222 ```python
223 >>> f.trace[10]
224 >>> f.trace[-2]
225 >>> f.trace[15:45]
226 >>> f.trace[:45:3]
227 ```
228
229* `header`
230
231 With addressing behaviour similar to `trace`, accessing items yield header
232 objects instead of numpy `ndarray`s. Headers are dict-like objects, where
233 keys are integers, seismic unix-style keys (in segyio.su module) and segyio
234 enums (segyio.TraceField).
235
236 Header values can be updated by assigning a dict-like to it, and keys not
237 present on the right-hand-side of the assignment are *unmodified*.
238
239 ```python
240 >>> f.header[5] = { segyio.su.tracl: 10 }
241 >>> f.header[5].items()
242 >>> f.header[5][25, 37] # read multiple values at once
243 ```
244
245* `iline`, `xline`
246
247 These modes will raise an error if the file is unstructured. They consider
248 arguments to `[]` as the *keys* of the respective lines. Line numbers are
249 always increasing, but can have arbitrary, uneven spacing. The valid names
250 can be found in the `ilines` and `xlines` properties.
251
252 As with traces, getting one line yields an `ndarray`, and a slice of lines
253 yields a generator of `ndarray`s. When using slices with a step, some
254 intermediate items might be skipped if it is not matched by the step, i.e.
255 doing `f.line[1:10:3]` on a file with lines `[1,2,3,4,5]` is equivalent of
256 looking up `1, 4, 7`, and finding `[1,4]`.
257
258 When working with a 4D pre-stack file, the first offset is implicitly read.
259 To access a different or a range of offsets, use comma separated indices or
260 ranges, as such: `f.iline[120, 4]`.
261
262* `fast`, `slow`
263
264 These are aliases for `iline` and `xline`, determined by how the traces are
265 laid out. For inline sorted files, `fast` would yield `iline`.
266
267* `depth_slice`
268
269 The depth slice is a horizontal, file-wide cut at a depth. The yielded
270 values are `ndarray`s and generators-of-arrays.
271
272* `gather`
273
274 The `gather` is the intersection of an inline and crossline, a vertical
275 column of the survey, and unless a single offset is specified returns an
276 offset x samples `ndarray`. In the presence of ranges, it returns a
277 generator of such `ndarray`s.
278
279* `text`
280
281 The `text` mode is an array of the textual headers, where `text[0]` is the
282 standard-mandated textual header, and `1..n` are the optional extended
283 headers.
284
285 The text headers are returned as 3200-byte string-like blobs (bytes in
286 Python 3, str in Python 2), as it is in the file. The `segyio.tools.wrap`
287 function can create a line-oriented version of this string.
288
289* `bin`
290
291 The values of the file-wide binary header with a dict-like interface.
292 Behaves like the `header` mode, but without the indexing.
293
294### Mode examples ###
295
296```python
297>>> for line in f.iline[:2430]:
298... print(np.average(line))
299
300>>> for line in f.xline[2:10]:
301... print(line)
302
303>>> for line in f.fast[::2]:
304... print(np.min(line))
305
306>>> for factor, offset in enumerate(f.iline[10, :]):
307... offset *= factor
308 print(offset)
309
310>>> f.gather[200, 241, :].shape
311
312>>> text = f.text[0]
313>>> type(text)
314<type 'bytes'> # 'str' in Python 2
315
316>>> f.trace[10] = np.zeros(len(f.samples))
317```
318
319More examples and recipes can be found in the docstrings `help(segyio)` and the
320[examples](#examples) section.
321
322## Project goals ##
323
324Segyio does not necessarily attempt to be the end-all of SEG-Y interactions;
325rather, we aim to lower the barrier to interacting with SEG-Y files for
326embedding, new applications or free-standing programs.
327
328Additionally, the aim is not to support the full standard or all exotic (but
329standard compliant) formatted files out there. Some assumptions are made, such
330as:
331
332 * All traces in a file are assumed to be of the same size
333
334Currently, segyio supports:
335 * Post-stack 3D volumes, sorted with respect to two header words (generally
336 INLINE and CROSSLINE)
337 * Pre-stack 4D volumes, sorted with respect to three header words (generally
338 INLINE, CROSSLINE, and OFFSET)
339 * Unstructured data, i.e. a collection of traces
340 * Most numerical formats (including IEEE 4- and 8-byte float, IBM float, 2-
341 and 4-byte integers)
342
343The writing functionality in segyio is largely meant to *modify* or adapt
344files. A file created from scratch is not necessarily a to-spec SEG-Y file, as
345we only necessarily write the header fields segyio needs to make sense of the
346geometry. It is still highly recommended that SEG-Y files are maintained and
347written according to specification, but segyio **does not** enforce this.
348
349
350### SEG-Y Revisions ###
351
352Segyio can handle a lot of files that are SEG-Y-like, i.e. segyio handles files
353that don't strictly conform to the SEG-Y standard. Segyio also does not
354discriminate between the revisions, but instead tries to use information
355available in the file. For an *actual* standard's reference, please see the
356publications by SEG:
357
358- [SEG-Y 0 (1975)](https://seg.org/Portals/0/SEG/News%20and%20Resources/Technical%20Standards/seg_y_rev0.pdf)
359- [SEG-Y 1 (2002)](https://seg.org/Portals/0/SEG/News%20and%20Resources/Technical%20Standards/seg_y_rev1.pdf)
360- [SEG-Y 2 (2017)](https://seg.org/Portals/0/SEG/News%20and%20Resources/Technical%20Standards/seg_y_rev2_0-mar2017.pdf)
361
362## Contributing ##
363
364We welcome all kinds of contributions, including code, bug reports, issues,
365feature requests, and documentation. The preferred way of submitting a
366contribution is to either make an
367[issue](https://github.com/equinor/segyio/issues) on github or by forking the
368project on github and making a pull request.
369
370## xarray integration ##
371
372[Alan Richardson](https://github.com/ar4) has written a great little tool for
373using [xarray](http://xarray.pydata.org/en/stable/) with segy files, which he
374demos in this
375[notebook](https://github.com/ar4/netcdf_segy/blob/master/notebooks/netcdf_segy.ipynb)
376
377## Reproducing the test data ##
378
379Small SEG-Y formatted files are included in the repository for test purposes.
380The data is non-sensical and made to be predictable, and it is reproducible by
381using segyio. The tests file are located in the test-data directory. To
382reproduce the data file, build segyio and run the test program `make-file.py`,
383`make-ps-file.py`, and `make-rotated-copies.py` as such:
384
385```python
386python examples/make-file.py small.sgy 50 1 6 20 25
387python examples/make-ps-file.py small-ps.sgy 10 1 5 1 4 1 3
388python examples/make-rotated-copies.py small.sgy
389```
390
391The small-lsb.sgy file was created by running the flip-endianness program. This
392program is included in the segyio source tree, but not a part of the package,
393and not intended for distribution and installation, only for reproducing test
394files.
395
396The seismic unix file small.su and small-lsb.su were created by the following
397commands:
398
399```bash
400segyread tape=small.sgy ns=50 remap=tracr,cdp byte=189l,193l conv=1 format=1 \
401 > small-lsb.su
402suswapbytes < small.su > small-lsb.su
403```
404
405If you have have small data files with a free license, feel free to submit it
406to the project!
407
408## Examples ##
409
410### Python ###
411
412Import useful libraries:
413
414```python
415import segyio
416import numpy as np
417from shutil import copyfile
418```
419
420Open segy file and inspect it:
421
422```python
423filename = 'name_of_your_file.sgy'
424with segyio.open(filename) as segyfile:
425
426 # Memory map file for faster reading (especially if file is big...)
427 segyfile.mmap()
428
429 # Print binary header info
430 print(segyfile.bin)
431 print(segyfile.bin[segyio.BinField.Traces])
432
433 # Read headerword inline for trace 10
434 print(segyfile.header[10][segyio.TraceField.INLINE_3D])
435
436 # Print inline and crossline axis
437 print(segyfile.xlines)
438 print(segyfile.ilines)
439```
440
441Read post-stack data cube contained in segy file:
442
443```python
444# Read data along first xline
445data = segyfile.xline[segyfile.xlines[1]]
446
447# Read data along last iline
448data = segyfile.iline[segyfile.ilines[-1]]
449
450# Read data along 100th time slice
451data = segyfile.depth_slice[100]
452
453# Read data cube
454data = segyio.tools.cube(filename)
455```
456
457Read pre-stack data cube contained in segy file:
458
459```python
460filename = 'name_of_your_prestack_file.sgy'
461with segyio.open(filename) as segyfile:
462
463 # Print offsets
464 print(segyfile.offset)
465
466 # Read data along first iline and offset 100: data [nxl x nt]
467 data = segyfile.iline[0, 100]
468
469 # Read data along first iline and all offsets gath: data [noff x nxl x nt]
470 data = np.asarray([np.copy(x) for x in segyfile.iline[0:1, :]])
471
472 # Read data along first 5 ilines and all offsets gath: data [noff nil x nxl x nt]
473 data = np.asarray([np.copy(x) for x in segyfile.iline[0:5, :]])
474
475 # Read data along first xline and all offsets gath: data [noff x nil x nt]
476 data = np.asarray([np.copy(x) for x in segyfile.xline[0:1, :]])
477```
478
479Read and understand fairly 'unstructured' data (e.g., data sorted in common shot gathers):
480
481```python
482filename = 'name_of_your_prestack_file.sgy'
483with segyio.open(filename, ignore_geometry=True) as segyfile:
484 segyfile.mmap()
485
486 # Extract header word for all traces
487 sourceX = segyfile.attributes(segyio.TraceField.SourceX)[:]
488
489 # Scatter plot sources and receivers color-coded on their number
490 plt.figure()
491 sourceY = segyfile.attributes(segyio.TraceField.SourceY)[:]
492 nsum = segyfile.attributes(segyio.TraceField.NSummedTraces)[:]
493 plt.scatter(sourceX, sourceY, c=nsum, edgecolor='none')
494
495 groupX = segyfile.attributes(segyio.TraceField.GroupX)[:]
496 groupY = segyfile.attributes(segyio.TraceField.GroupY)[:]
497 nstack = segyfile.attributes(segyio.TraceField.NStackedTraces)[:]
498 plt.scatter(groupX, groupY, c=nstack, edgecolor='none')
499```
500
501Write segy file using same header of another file but multiply data by *2
502
503```python
504input_file = 'name_of_your_input_file.sgy'
505output_file = 'name_of_your_output_file.sgy'
506
507copyfile(input_file, output_file)
508
509with segyio.open(output_file, "r+") as src:
510
511 # multiply data by 2
512 for i in src.ilines:
513 src.iline[i] = 2 * src.iline[i]
514```
515
516[Make segy file from sctrach](python/examples/make-file.py)
517
518### MATLAB ###
519
520```
521filename='name_of_your_file.sgy'
522
523% Inspect segy
524Segy_struct=SegySpec(filename,189,193,1);
525
526% Read headerword inline for each trace
527Segy.get_header(filename,'Inline3D')
528
529%Read data along first xline
530data= Segy.readCrossLine(Segy_struct,Segy_struct.crossline_indexes(1));
531
532%Read cube
533data=Segy.get_cube(Segy_struct);
534
535%Write segy, use same header but multiply data by *2
536input_file='input_file.sgy';
537output_file='output_file.sgy';
538copyfile(input_file,output_file)
539data = Segy.get_traces(input_file);
540data1 = 2*data;
541Segy.put_traces(output_file, data1);
542```
543
544## Common issues ##
545
546### Creating a new file is very slow, or copying headers is slow ###
547
548Quite often issues show up where someone struggle with the performance of
549segyio, in particular when creating new files. The culprit is often this code:
550
551```
552with segyio.create('new.sgy', spec) as dst:
553 dst.header = headers
554```
555
556The code itself is perfectly ok, but it has subtle behaviour on some systems
557when the file is newly created: it is performing many scattered writes to a
558[sparse file](https://en.wikipedia.org/wiki/Sparse_file). This can be fast or
559slow, largely depending on the [file
560system](https://github.com/equinor/segyio/issues/423).
561
562#### Possible solutions
563
564Rewrite the loop to write to the file contiguously:
565```
566with segyio.create('new.sgy', spec) as dst:
567 for i in range(spec.tracecount):
568 dst.header[i] = headers[i]
569 dst.trace[i] = traces[i]
570```
571
572If the file is modified copy of another file, without changing the trace
573lengths, it's often faster (and easier!) to first copy the file without segyio,
574and then use segyio to modify the copy in-place:
575
576```
577shutil.copyfile(srcfile, dstfile)
578with segyio.open(dstfile) as f:
579 f.header = headers
580```
581
582### ImportError: libsegyio.so.1: cannot open shared object file
583
584This error shows up when the loader cannot find the core segyio library. If
585you've explicitly set the install prefix (with `-DCMAKE_INSTALL_PREFIX`) you
586must configure your loader to also look in this prefix, either with a
587`ld.conf.d` file or the `LD_LIBRARY_PATH` variable.
588
589If you haven't set `CMAKE_INSTALL_PREFIX`, cmake will by default install to
590`/usr/local`, which your loader usually knows about. On Debian based systems,
591the library often gets installed to `/usr/local/lib`, which the loader may not
592know about. See [issue #239](https://github.com/equinor/segyio/issues/239).
593
594#### Possible solutions
595
596* Configure the loader (`sudo ldconfig` often does the trick)
597* Install with a different, known prefix, e.g. `-DCMAKE_INSTALL_LIBDIR=lib64`
598
599### RuntimeError: unable to find sorting
600
601This exception is raised when segyio tries to open the in strict mode, under
602the assumption that the file is a regular, sorted 3D volume. If the file is
603just a collection of traces in arbitrary order, this would fail.
604
605#### Possible solutions
606
607Check if segyio.open `iline` and `xline` input parameters are correct for current file.
608Segyio supports files that are just a collection of traces too, but has to be
609told that it's ok to do so. Pass `strict = False` or `ignore_geometry = True`
610to `segyio.open` to allow or force unstructured mode respectively. Please note
611that `f.iline` and similar features are now disabled and will raise errors.
612
613## History ##
614Segyio was initially written and is maintained by [Equinor
615ASA](http://www.equinor.com/) as a free, simple, easy-to-use way of interacting
616with seismic data that can be tailored to our needs, and as contribution to the
617free software community.
618