1.\" $OpenBSD: mirroring-ports.7,v 1.27 2015/03/13 19:58:41 jmc Exp $ 2.\" 3.\" Copyright (c) 2000,2012 Marc Espie 4.\" 5.\" All rights reserved. 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 1. Redistributions of source code must retain the above copyright 11.\" notice, this list of conditions and the following disclaimer. 12.\" 2. Redistributions in binary form must reproduce the above copyright 13.\" notice, this list of conditions and the following disclaimer in the 14.\" documentation and/or other materials provided with the distribution. 15.\" 16.\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR 17.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 18.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 19.\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT, 20.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 21.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 22.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 23.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 24.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 25.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 26.\" 27.Dd $Mdocdate: March 13 2015 $ 28.Dt MIRRORING-PORTS 7 29.Os 30.Sh NAME 31.Nm mirroring-ports 32.Nd how to build a mirror for ports distfiles 33.Sh DESCRIPTION 34The 35.Nm OpenBSD Ports Collection 36offers some powerful tools to mirror software sources. 37.Pp 38.Xr dpb 1 39features a 40.Fl F 41option which is explicitly designed for mirroring distfiles. 42.Pp 43If run with 44.Fl F Ar jobs , 45.Nm dpb 46will 47.Bl -bullet 48.It 49Limit itself to fetching distfiles, and not build any packages. 50.It 51Disregard any architecture or broken annotation, and try to fetch every 52distfile. 53.It 54Fetch files to a temporary copy named 55.Pa some_file.part 56using 57.Ic ftp -C 58to resume interrupted downloads. 59.It 60Keep a global list of sha256 checksums as 61.Pa ${DISTDIR}/distinfo , 62and use that to refetch files when the ports tree records a changing checksum. 63.It 64Produces a log of old distfiles in 65.Pa ${DISTDIR}/history , 66.It 67Create sha256 links under 68.Pa ${DISTDIR}/by_cipher/sha256 69as per 70.Xr link-checksum 1 Ns 's 71former duties. 72.El 73.Pp 74For partial distfiles collections, 75.Nm dpb 76can also be run with 77.Fl D Ns Ar HISTORY_ONLY 78to scan the full ports tree and update 79.Pa ${DISTDIR}/history 80without fetching anything. 81.Sh FILES 82.Bl -tag -width toto 83.It Pa ${DISTDIR}/distinfo 84a cache of known distfiles with their respective checksums. 85.It Pa ${DISTDIR}/history 86List of files appearing in 87.Pa ${DISTDIR}/distinfo 88that seem to no longer be required by the ports tree. 89.Xr dpb 1 90will append to this file each time it is run on the whole tree 91(option 92.Fl a ) 93and only if the ports tree scan finishes without error. 94Each line is of the form 95.Bd -literal -offset indent 96timestamp SHA256 (file) = sha 97.Ed 98.Pp 99denoting the first fime a file/sha entry was no longer seen in the ports tree. 100.El 101.Sh SEE ALSO 102.Xr clean-old-distfiles 1 , 103.Xr dpb 1 , 104.Xr ports 7 105.Sh HISTORY 106The new integrated 107.Fl F 108option to 109.Xr dpb 1 110was introduced in 111.Ox 5.1 , 112replacing the original infrastructure introduced in 113.Ox 2.7 . 114.Sh CAVEATS 115Changing checksums is a recurring problem that is outside the direct 116control of the 117.Ox 118Project. 119Some software distributors change distribution files without 120warning, without changing the file name proper. 121Once the problem has been identified, the port maintainer should usually 122contact the software author to fix the problem or, if the software author 123is unresponsive, the maintainer should use 124.Ev DIST_SUBDIR 125to provide some state to guard against shifting checksums. 126.Pp 127However, a more robust approach is also needed, so that ports users can 128depend on distfiles mirrors to carry what they need irrespective of those 129synchronization issues. 130The 131.Pa ${DISTFILES}/by-cipher/sha256 132directory provides more persistent access to the distfiles, 133indexed through the actual checksums that the files should match. 134Provided mirroring is run sufficiently often, 135two versions of the same distfile with respective checksums cksum1 and cksum2 136will be available under the names 137.Pa ${DISTFILES}/sha256/c1/cksum1/distfile 138and 139.Pa ${DISTFILES}/sha256/c2/cksum2/distfile . 140.Pp 141If 142.Ev REFETCH 143is set to true, 144.Xr bsd.port.mk 5 145will try to retrieve files under that naming scheme as a last resort. 146