camcontrol: add support for Toshiba drive firmware update

Thanks to Toshiba for providing the SCSI spec for their latest
generation drives so I could confirm how they operate.

The firmware download

camcontrol: add support for Toshiba drive firmware update

Thanks to Toshiba for providing the SCSI spec for their latest
generation drives so I could confirm how they operate.

The firmware download works in a pretty standard way, so this
is a straightforward table addition.

sbin/camcontrol/camcontrol.8:
Document that Toshiba drives are supported for fwdownload,
and that it was tested on TOSHIBA MG10SFA22TE 22TB drives.

sbin/camcontrol/fwdownload.c:
Add TOSHIBA to the known SCSI vendors list for fwdownload.

Sponsored by: Spectra Logic
MFC after: 3 days

show more ...

sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remo

sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

show more ...

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

camcontrol: Migrate to modern uintXX_t from u_intXX_t

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t.

MFC After: 3 days
Sponsored by: Netfl

camcontrol: Migrate to modern uintXX_t from u_intXX_t

As per https://lists.freebsd.org/archives/freebsd-scsi/2023-July/000257.html
move to the modern uintXX_t.

MFC After: 3 days
Sponsored by: Netflix

show more ...

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

show more ...

camcontrol fwdownload minor improvements

Minor improvements to the fwdownload code suggested by chs@:
o Print the path_id/target we're rescanning so it's not invisible
o No need for XPT_GDEVLIST, al

camcontrol fwdownload minor improvements

Minor improvements to the fwdownload code suggested by chs@:
o Print the path_id/target we're rescanning so it's not invisible
o No need for XPT_GDEVLIST, all the info is filled in. Remove sending it
as well as a comment related to it from a mistaken observation. libcam
always fills these in properly, so use those for the ccb path/target.
o Don't leak /dev/xpt fd in success cases.
o Rename fw_rescan_lun to fw_rescan_target and pass sim_mode to
only print path_id and target_id info.

Reviewed by: chs@
Fixes: 9835900cb95bcd068774934961fb1419719d595b
Sponsored by: Netflix
MFC After: 1 week
Differential Revision: https://reviews.freebsd.org/D34348

show more ...

camcontrol: Force a rescan of the lun after firmware download.

After downloading the firmware to a device, it's inquiry data likely
will change. Force a rescan of the target with the CAM_EXPECT_INQ_

camcontrol: Force a rescan of the lun after firmware download.

After downloading the firmware to a device, it's inquiry data likely
will change. Force a rescan of the target with the CAM_EXPECT_INQ_CHANGE
flag to get it to record the new inqury data as being expected. This
avoids the need for a 'camcontrol rescan' on the device which detaches
and re-attaches the disk (da, ada) device. This brings fwdownload up to
nvmecontrol's ability to do the same thing w/o changing the exposed
nvme/nvd/nda device. We scan the target and not the LUN because dual
actuator drives have multiple LUNs, but the firmware is global across
many vendors' drives (and the so far theoretical ones that aren't won't
be harmed by the rescan).

Since the underlying struct disk is now preserved accross this
operation, it's now possible to upgrade firmware of a root device w/o
crashing the system. On systems that are quite busy, the worst that
happens is that certain operaions are reported cancelled when the new
firmware is activated. These operations are retried with the normal CAM
recovery mechanisms and will work on the retry. The only visible hiccup
is the time that new firmware is flashing / initializing. One should not
consider this operation completely risk free, however, since not all
drives are well behaved after a firmware download.

MFC After: 1 week
Relnotes: yes
Sponsored by: Netflix
Feedback by: mav
Differential Revision: https://reviews.freebsd.org/D34325

show more ...

camcontrol(8): remove unnecessary CCB zeroing

After 3e404b8c53d, cam_getccb(3) clears the returned CCB, making
a number of calls to CCB_CLEAR_ALL_EXCEPT_HDR(3) unnecessary.

Reviewed By: imp
Sponsor

camcontrol(8): remove unnecessary CCB zeroing

After 3e404b8c53d, cam_getccb(3) clears the returned CCB, making
a number of calls to CCB_CLEAR_ALL_EXCEPT_HDR(3) unnecessary.

Reviewed By: imp
Sponsored by: NetApp, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D27812

show more ...

Supply SAT layer with valid transfer sizes.

This is a rework of r344701, that noticed that number of bytes passes to
8 bit sector count field gets truncated. First decision was to not pass
anything

Supply SAT layer with valid transfer sizes.

This is a rework of r344701, that noticed that number of bytes passes to
8 bit sector count field gets truncated. First decision was to not pass
anything, since ATA specs define the field as N/A. But it appeared to be a
problem for some SAT devices, that require information about data transfer
to operate properly. Some additional investigation shown that it is quite
a common practice to set unused fields of ATA commands (fortunately ATA
specs formally allow it) to supply the information to SAT layer. I have
found SAS-SATA interposer that does not allow pass-through without it.

As side effect, reduce code duplication by removing ata_do_28bit_cmd()
function, replacing it with more universal ata_do_cmd().

MFC after: 1 week
Sponsored by: iXsystems, Inc.

show more ...

Use the more proper term of SATL instead of ATA_BEHIND_SCSI.

Most people know SAS attached SATA devices by the name SAT or SATL
(with the latter being a little more common). Change the device type
A

Use the more proper term of SATL instead of ATA_BEHIND_SCSI.

Most people know SAS attached SATA devices by the name SAT or SATL
(with the latter being a little more common). Change the device type
ATA_BEHIND_SCSI to SATL since it's more specific and meaningful.

Suggested by: scottl@

show more ...

Make WD and WDC aliases for HGST.

HGST was bought by WDC. Over the years, it has sold different drives
branded as HGST, WD or WDC. All of them need the HGST workaround of
sending 4k-sized packets (o

Make WD and WDC aliases for HGST.

HGST was bought by WDC. Over the years, it has sold different drives
branded as HGST, WD or WDC. All of them need the HGST workaround of
sending 4k-sized packets (or multiples of 4k). And the ones that don't
really need this aren't broken by this change. Submitter is the vendor
who has tested these changes on a number of drives. I've simplified it
slightly, since we don't need additional vendors for this at this
time.

Submitted by: JacobBurley via github on behalf of WDC
Pull Request: https://github.com/freebsd/freebsd/pull/391

show more ...

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

show more ...

Fix memory leaks in camcontrol

Reported by: Coverity
CID: 1331674, 1331675
Reviewed by: ken
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D1

Fix memory leaks in camcontrol

Reported by: Coverity
CID: 1331674, 1331675
Reviewed by: ken
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D10588

show more ...

Add task attribute support to camcontrol(8).

Users can use the new generic argument, -Q task_attr, to specify a task
attribute (simple, ordered, head of queue, aca) for the commands issued.
The the

Add task attribute support to camcontrol(8).

Users can use the new generic argument, -Q task_attr, to specify a task
attribute (simple, ordered, head of queue, aca) for the commands issued.
The the default is simple, which works with all SCSI devices that support
tagged queueing.

This will mostly be useful for debugging target behavior in certain
situations.

You can try it out by compiling CTL with CTL_IO_DELAY turned on (in
sys/cam/ctl/ctl_io.h) and then do something like this with one of the CTL
LUNs:

ctladm delay 0:0 -l done -t 10
camcontrol tur da34 -v

And at then before the 10 second timer is up, in another terminal:

camcontrol inquiry da34 -Q ordered -v

The Inquiry should complete just after the TUR completes. Ordinarily
it would complete first because of the delay injection, but because the
task attribute is set to ordered in this case, CTL holds it up until the
previous command has completed.

sbin/camcontrol/camcontrol.c:
Add the new generic argument, -Q, which allows the user to specify
a SCSI task attribute. The user can specify task attributes by
name or numerically.

Add a new task_attr arguments to SCSI sub-functions.

sbin/camcontrol/attrib.c,
sbin/camcontrol/camcontrol.h,
sbin/camcontrol/fwdownload.c,
sbin/camcontrol/modeedit.c,
sbin/camcontrol/persist.c,
sbin/camcontrol/timestamp.c,
sbin/camcontrol/zone.c:
Add the new task_attr argument to SCSI sub-functions.

sbin/camcontrol/camcontrol.8:
Document the new -Q option, and add an example.

Sponsored by: Spectra Logic
MFC after: 1 week

show more ...

Fix multiple Coverity Out-of-bounds access false postive issues in CAM

The currently used idiom for clearing the part of a ccb after its
header generates one or two Coverity errors for each time it

Fix multiple Coverity Out-of-bounds access false postive issues in CAM

The currently used idiom for clearing the part of a ccb after its
header generates one or two Coverity errors for each time it is
used. All instances generate an Out-of-bounds access (ARRAY_VS_SINGLETON)
error because of the treatment of the header as a two element array,
with a pointer to the non-existent second element being passed as
the starting address to bzero(). Some instances also alsp generate
Out-of-bounds access (OVERRUN) errors, probably because the space
being cleared is larger than the sizeofstruct ccb_hdr).

In addition, this idiom is difficult for humans to understand and
it is error prone. The user has to chose the proper struct ccb_*
type (which does not appear in the surrounding code) for the sizeof()
in the length calculation. I found several instances where the
length was incorrect, which could cause either an actual out of
bounds write, or incompletely clear the ccb.

A better way is to write the code to clear the ccb itself starting
at sizeof(ccb_hdr) bytes from the start of the ccb, and calculate
the length based on the specific type of struct ccb_* being cleared
as specified by the union ccb member being used. The latter can
normally be seen in the nearby code. This is friendlier for Coverity
and other static analysis tools because they will see that the
intent is to clear the trailing part of the ccb.

Wrap all of the boilerplate code in a convenient macro that only
requires a pointer to the desired union ccb member (or a pointer
to the union ccb itself) as an argument.

Reported by: Coverity
CID: 1007578, 1008684, 1009724, 1009773, 1011304, 1011306
CID: 1011307, 1011308, 1011309, 1011310, 1011311, 1011312
CID: 1011313, 1011314, 1011315, 1011316, 1011317, 1011318
CID: 1011319, 1011320, 1011321, 1011322, 1011324, 1011325
CID: 1011326, 1011327, 1011328, 1011329, 1011330, 1011374
CID: 1011390, 1011391, 1011392, 1011393, 1011394, 1011395
CID: 1011396, 1011397, 1011398, 1011399, 1011400, 1011401
CID: 1011402, 1011403, 1011404, 1011405, 1011406, 1011408
CID: 1011409, 1011410, 1011411, 1011412, 1011413, 1011414
CID: 1017461, 1018387, 1086860, 1086874, 1194257, 1229897
CID: 1229968, 1306229, 1306234, 1331282, 1331283, 1331294
CID: 1331295, 1331535, 1331536, 1331539, 1331540, 1341623
CID: 1341624, 1341637, 1341638, 1355264, 1355324
Reviewed by: scottl, ken, delphij, imp
MFH: 1 month
Differential Revision: https://reviews.freebsd.org/D6496

show more ...

Add support for managing Shingled Magnetic Recording (SMR) drives.

This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which

Add support for managing Shingled Magnetic Recording (SMR) drives.

This change includes support for SCSI SMR drives (which conform to the
Zoned Block Commands or ZBC spec) and ATA SMR drives (which conform to
the Zoned ATA Command Set or ZAC spec) behind SAS expanders.

This includes full management support through the GEOM BIO interface, and
through a new userland utility, zonectl(8), and through camcontrol(8).

This is now ready for filesystems to use to detect and manage zoned drives.
(There is no work in progress that I know of to use this for ZFS or UFS, if
anyone is interested, let me know and I may have some suggestions.)

Also, improve ATA command passthrough and dispatch support, both via ATA
and ATA passthrough over SCSI.

Also, add support to camcontrol(8) for the ATA Extended Power Conditions
feature set. You can now manage ATA device power states, and set various
idle time thresholds for a drive to enter lower power states.

Note that this change cannot be MFCed in full, because it depends on
changes to the struct bio API that break compatilibity. In order to
avoid breaking the stable API, only changes that don't touch or depend on
the struct bio changes can be merged. For example, the camcontrol(8)
changes don't depend on the new bio API, but zonectl(8) and the probe
changes to the da(4) and ada(4) drivers do depend on it.

Also note that the SMR changes have not yet been tested with an actual
SCSI ZBC device, or a SCSI to ATA translation layer (SAT) that supports
ZBC to ZAC translation. I have not yet gotten a suitable drive or SAT
layer, so any testing help would be appreciated. These changes have been
tested with Seagate Host Aware SATA drives attached to both SAS and SATA
controllers. Also, I do not have any SATA Host Managed devices, and I
suspect that it may take additional (hopefully minor) changes to support
them.

Thanks to Seagate for supplying the test hardware and answering questions.

sbin/camcontrol/Makefile:
Add epc.c and zone.c.

sbin/camcontrol/camcontrol.8:
Document the zone and epc subcommands.

sbin/camcontrol/camcontrol.c:
Add the zone and epc subcommands.

Add auxiliary register support to build_ata_cmd(). Make sure to
set the CAM_ATAIO_NEEDRESULT, CAM_ATAIO_DMA, and CAM_ATAIO_FPDMA
flags as appropriate for ATA commands.

Add a new get_ata_status() function to parse ATA result from SCSI
sense descriptors (for ATA passthrough over SCSI) and ATA I/O
requests.

sbin/camcontrol/camcontrol.h:
Update the build_ata_cmd() prototype

Add get_ata_status(), zone(), and epc().

sbin/camcontrol/epc.c:
Support for ATA Extended Power Conditions features. This includes
support for all features documented in the ACS-4 Revision 12
specification from t13.org (dated February 18, 2016).

The EPC feature set allows putting a drive into a power power mode
immediately, or setting timeouts so that the drive will
automatically enter progressively lower power states after various
idle times.

sbin/camcontrol/fwdownload.c:
Update the firmware download code for the new build_ata_cmd()
arguments.

sbin/camcontrol/zone.c:
Implement support for Shingled Magnetic Recording (SMR) drives
via SCSI Zoned Block Commands (ZBC) and ATA Zoned Device ATA
Command Set (ZAC).

These specs were developed in concert, and are functionally
identical. The primary differences are due to SCSI and ATA
differences. (SCSI is big endian, ATA is little endian, for
example.)

This includes support for all commands defined in the ZBC and
ZAC specs.

sys/cam/ata/ata_all.c:
Decode a number of additional ATA command names in ata_op_string().

Add a new CCB building function, ata_read_log().

Add ata_zac_mgmt_in() and ata_zac_mgmt_out() CCB building
functions. These support both DMA and NCQ encapsulation.

sys/cam/ata/ata_all.h:
Add prototypes for ata_read_log(), ata_zac_mgmt_out(), and
ata_zac_mgmt_in().

sys/cam/ata/ata_da.c:
Revamp the ada(4) driver to support zoned devices.

Add four new probe states to gather information needed for zone
support.

Add a new adasetflags() function to avoid duplication of large
blocks of flag setting between the async handler and register
functions.

Add new sysctl variables that describe zone support and paramters.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

sys/cam/scsi/scsi_all.c:
Add command descriptions for the ZBC IN/OUT commands.

Add descriptions for ZBC Host Managed devices.

Add a new function, scsi_ata_pass() to do ATA passthrough over
SCSI. This will eventually replace scsi_ata_pass_16() -- it
can create the 12, 16, and 32-byte variants of the ATA
PASS-THROUGH command, and supports setting all of the
registers defined as of SAT-4, Revision 5 (March 11, 2016).

Change scsi_ata_identify() to use scsi_ata_pass() instead of
scsi_ata_pass_16().

Add a new scsi_ata_read_log() function to facilitate reading
ATA logs via SCSI.

sys/cam/scsi/scsi_all.h:
Add the new ATA PASS-THROUGH(32) command CDB. Add extended and
variable CDB opcodes.

Add Zoned Block Device Characteristics VPD page.

Add ATA Return SCSI sense descriptor.

Add prototypes for scsi_ata_read_log() and scsi_ata_pass().

sys/cam/scsi/scsi_da.c:
Revamp the da(4) driver to support zoned devices.

Add five new probe states, four of which are needed for ATA
devices.

Add five new sysctl variables that describe zone support and
parameters.

The da(4) driver supports SCSI ZBC devices, as well as ATA ZAC
devices when they are attached via a SCSI to ATA Translation (SAT)
layer. Since ZBC -> ZAC translation is a new feature in the T10
SAT-4 spec, most SATA drives will be supported via ATA commands
sent via the SCSI ATA PASS-THROUGH command. The da(4) driver will
prefer the ZBC interface, if it is available, for performance
reasons, but will use the ATA PASS-THROUGH interface to the ZAC
command set if the SAT layer doesn't support translation yet.
As I mentioned above, ZBC command support is untested.

Add support for the new BIO_ZONE bio, and all of its subcommands:
DISK_ZONE_OPEN, DISK_ZONE_CLOSE, DISK_ZONE_FINISH, DISK_ZONE_RWP,
DISK_ZONE_REPORT_ZONES, and DISK_ZONE_GET_PARAMS.

Add scsi_zbc_in() and scsi_zbc_out() CCB building functions.

Add scsi_ata_zac_mgmt_out() and scsi_ata_zac_mgmt_in() CCB/CDB
building functions. Note that these have return values, unlike
almost all other CCB building functions in CAM. The reason is
that they can fail, depending upon the particular combination
of input parameters. The primary failure case is if the user
wants NCQ, but fails to specify additional CDB storage. NCQ
requires using the 32-byte version of the SCSI ATA PASS-THROUGH
command, and the current CAM CDB size is 16 bytes.

sys/cam/scsi/scsi_da.h:
Add ZBC IN and ZBC OUT CDBs and opcodes.

Add SCSI Report Zones data structures.

Add scsi_zbc_in(), scsi_zbc_out(), scsi_ata_zac_mgmt_out(), and
scsi_ata_zac_mgmt_in() prototypes.

sys/dev/ahci/ahci.c:
Fix SEND / RECEIVE FPDMA QUEUED in the ahci(4) driver.

ahci_setup_fis() previously set the top bits of the sector count
register in the FIS to 0 for FPDMA commands. This is okay for
read and write, because the PRIO field is in the only thing in
those bits, and we don't implement that further up the stack.

But, for SEND and RECEIVE FPDMA QUEUED, the subcommand is in that
byte, so it needs to be transmitted to the drive.

In ahci_setup_fis(), always set the the top 8 bits of the
sector count register. We need it in both the standard
and NCQ / FPDMA cases.

sys/geom/eli/g_eli.c:
Pass BIO_ZONE commands through the GELI class.

sys/geom/geom.h:
Add g_io_zonecmd() prototype.

sys/geom/geom_dev.c:
Add new DIOCZONECMD ioctl, which allows sending zone commands to
disks.

sys/geom/geom_disk.c:
Add support for BIO_ZONE commands.

sys/geom/geom_disk.h:
Add a new flag, DISKFLAG_CANZONE, that indicates that a given
GEOM disk client can handle BIO_ZONE commands.

sys/geom/geom_io.c:
Add a new function, g_io_zonecmd(), that handles execution of
BIO_ZONE commands.

Add permissions check for BIO_ZONE commands.

Add command decoding for BIO_ZONE commands.

sys/geom/geom_subr.c:
Add DDB command decoding for BIO_ZONE commands.

sys/kern/subr_devstat.c:
Record statistics for REPORT ZONES commands. Note that the
number of bytes transferred for REPORT ZONES won't quite match
what is received from the harware. This is because we're
necessarily counting bytes coming from the da(4) / ada(4) drivers,
which are using the disk_zone.h interface to communicate up
the stack. The structure sizes it uses are slightly different
than the SCSI and ATA structure sizes.

sys/sys/ata.h:
Add many bit and structure definitions for ZAC, NCQ, and EPC
command support.

sys/sys/bio.h:
Convert the bio_cmd field to a straight enumeration. This will
yield more space for additional commands in the future. After
change r297955 and other related changes, this is now possible.
Converting to an enumeration will also prevent use as a bitmask
in the future.

sys/sys/disk.h:
Define the DIOCZONECMD ioctl.

sys/sys/disk_zone.h:
Add a new API for managing zoned disks. This is very close to
the SCSI ZBC and ATA ZAC standards, but uses integers in native
byte order instead of big endian (SCSI) or little endian (ATA)
byte arrays.

This is intended to offer to the complete feature set of the ZBC
and ZAC disk management without requiring the application developer
to include SCSI or ATA headers. We also use one set of headers
for ioctl consumers and kernel bio-level consumers.

sys/sys/param.h:
Bump __FreeBSD_version for sys/bio.h command changes, and inclusion
of SMR support.

usr.sbin/Makefile:
Add the zonectl utility.

usr.sbin/diskinfo/diskinfo.c
Add disk zoning capability to the 'diskinfo -v' output.

usr.sbin/zonectl/Makefile:
Add zonectl makefile.

usr.sbin/zonectl/zonectl.8
zonectl(8) man page.

usr.sbin/zonectl/zonectl.c
The zonectl(8) utility. This allows managing SCSI or ATA zoned
disks via the disk_zone.h API. You can report zones, reset write
pointers, get parameters, etc.

Sponsored by: Spectra Logic
Differential Revision: https://reviews.freebsd.org/D6147
Reviewed by: wblock (documentation)

show more ...

camcontrol(8): Fix another trivial double-free

Reported by: Coverity
CID: 1331222
Sponsored by: EMC / Isilon Storage Division

sbin: minor spelling fixes.

No functional change.

Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.

The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

When the user's filename is

Revamp camcontrol(8) fwdownload support and add the opcodes subcommand.

The significant changes and bugs fixed here are:

1. Fixed a bug in the progress display code:

When the user's filename is too big, or his terminal width is too
small, the progress code could wind up using a negative number for
the length of the "stars" that it uses to indicate progress.

This negative value was assigned to an unsigned variable, resulting
in a very large positive value.

The result is that we wound up writing garbage from memory to the
user's terminal.

With an 80 column terminal, a file name length of more than 35
characters would generate this problem.

To address this, we now set a minimum progress bar length, and
truncate the user's file name as needed.

This has been tested with large filenames and small terminals, and
at least produces reasonable results. If the terminal is too
narrow, the progress display takes up an additional line with each
update, but this is more user friendly than writing garbage to the
tty.

2. SATA drives connected via a SATA controller didn't have SCSI Inquiry
data populated in struct cam_device. This meant that the code in
fw_get_vendor() in fwdownload.c would try to match a zero-length
vendor ID, and so return the first entry in the vendor table. (Which
used to be HITACHI.) Fixed by grabbing identify data, passing the
identify buffer into fw_get_vendor(), and matching against the model
name.

3. SATA drives connected via a SAS controller do have Inquiry data
populated. The table included a couple of entries -- "ATA ST" and
"ATA HDS", intended to handle Seagate and Hitachi SATA drives attached
via a SAS controller. SCSI to ATA translation layers use a vendor
ID of "ATA" (which is standard), and then the model name from the ATA
identify data as the SCSI product name when they are returning data on
SATA disks. The cam_strmatch code will match the first part of the
string (because the length it is given is the length of the vendor,
"ATA"), and return 0 (i.e. a match). So all SATA drives attached to
a SAS controller would be programmed using the Seagate method
(WRITE BUFFER mode 7) of SCSI firmware downloading.

4. Issue #2 above covered up a bug in fw_download_img() -- if the
maximum packet size in the vendor table was 0, it tried to default
to a packet size of 32K. But then it didn't actually succeed in
doing that, because it set the packet size to the value that was
in the vendor table (0). Now that we actually have ATA attached
drives fall use the VENDOR_ATA case, we need a reasonable default
packet size. So this is fixed to properly set the default packet size.

5. Add support for downloading firmware to IBM LTO drives, and add a
firmware file validation method to make sure that the firmware
file matches the drive type. IBM tape drives include a Load ID and
RU name in their vendor-specific VPD page 0x3. Those should match
the IDs in the header of the firmware file to insure that the
proper firmware file is loaded.

6. This also adds a new -q option to the camcontrol fwdownload
subcommand to suppress informational output. When -q is used in
combination with -y, the firmware upgrade will happen without
prompting and without output except if an error condition occurs.

7. Re-add support for printing out SCSI inquiry information when
asking the user to confirm that they want to download firmware, and
add printing of ATA Identify data if it is a SATA disk. This was
removed in r237281 when support for flashing ATA disks was added.

8. Add a new camcontrol(8) "opcodes" subcommand, and use the
underlying code to get recommended timeout values for drive
firmware downloads.

Many SCSI devices support the REPORT SUPPORTED OPERATION CODES
command, and some support the optional timeout descriptor that
specifies nominal and recommended timeouts for the commands
supported by the device.

The new camcontrol opcodes subcommand allows displaying all
opcodes supported by a drive, information about which fields
in a SCSI CDB are actually used by a given SCSI device, and the
nominal and recommended timeout values for each command.

Since firmware downloads can take a long time in some devices, and
the time varies greatly between different types of devices, take
advantage of the infrastructure used by the camcontrol opcodes
subcommand to determine the best timeout to use for the WRITE
BUFFER command in SCSI device firmware downloads.

If the device recommends a timeout, it is likely to be more
accurate than the default 50 second timeout used by the firmware
download code. If the user specifies a timeout, it will override
the default or device recommended timeout. If the device doesn't
support timeout descriptors, we fall back to the default.

9. Instead of downloading firmware to SATA drives behind a SAS controller
using WRITE BUFFER, use the SCSI ATA PASS-THROUGH command to compose
an ATA DOWNLOAD MICROCODE command and it to the drive. The previous
version of this code attempted to send a SCSI WRITE BUFFER command to
SATA drives behind a SAS controller. Although that is part of the
SAT-3 spec, it doesn't work with the parameters used with LSI
controllers at least.

10.Add a new mechanism for making common ATA passthrough and
ATA-behind-SCSI passthrough commands.

The existing camcontrol(8) ATA command mechanism checks the device
type on every command executed. That works fine for individual
commands, but is cumbersome for things like a firmware download
that send a number of commands.

The fwdownload code detects the device type up front, and then
sends the appropriate commands.

11.In simulation mode (-s), if the user specifies the -v flag, print out
the SCSI CDB or ATA registers that would be sent to the drive. This will
aid in debugging any firmware download issues.

sbin/camcontrol/fwdownload.c:
Add a device type to the fw_vendor structure, so that we can
specify different download methods for different devices from the
same vendor. In this case, IBM hard drives (from when they
still made hard drives) and tape drives.

Add a tur_status field to the fw_vendor structure so that we can
specify whether the drive to be upgraded should be ready, not
ready, or whether it doesn't matter. Add the corresponding
capability in fw_download_img().

Add comments describing each of the vendor table fields.

Add HGST and SmrtStor to the supported SCSI vendors list.

In fw_get_vendor(), look at ATA identify data if we have a SATA
device to try to identify what the drive vendor is.

Add IBM firmware file validation. This gets VPD page 0x3, and
compares the Load ID and RU name in the page to the values
included in the header. The validation code will refuse to load
a firmware file if the values don't match. This does allow the
user to attempt a downgrade; whether or not it succeeds will
likely depend on the drive settings.

Add a -q option, and disable all informative output
(progress bars, etc.) when this is enabled.

Re-add the inquiry in the confirmation dialog so the user has
a better idea of which device he is talking to. Add support for
displaying ATA identify data.

Don't automatically disable confirmation in simulation (-s) mode.
This allows the user to see the inquiry or identify data in the
dialog, and see exactly what they would see when the command
actually runs. Also, in simulation mode, if the user specifies
the -v flag, print out the SCSI CDB or ATA registers that would
be sent to the drive. This will aid in debugging any firmware
download issues.

Add a timeout field and timeout type to the firmware download
vendor table. This allows specifying a default timeout and allows
specifying whether we should attempt to probe for a recommended
timeout from the drive.

Add a new fuction, fw_get_timeout(), that will determine
which timeout to use for the WRITE BUFFER command. If the
user specifies a timeout, we always use that. Otherwise,
we will use the drive recommended timeout, if available,
and fall back to the default when a drive recommended
timeout isn't available.

When we prompt the user, tell him what timeout we're going
to use, and the source of the timeout.

Revamp the way SATA devices are handled.

In fwdownload(), use the new get_device_type() function to
determine what kind of device we're talking to.

Allow firmware downloads to any SATA device, but restrict
SCSI downloads to known devices. (The latter is not a
change in behavior.)

Break out the "ready" check from fw_download_img() into a
new subfunction, fw_check_device_ready(). This sends the
appropriate command to the device in question -- a TEST
UNIT READY or an IDENTIFY. The IDENTIFY for SATA devices
a SAT layer is done using the SCSI ATA PASS-THROUGH
command.

Use the new build_ata_cmd() function to build either a SCSI or
ATA I/O CCB to issue the DOWNLOAD MICROCODE command to SATA
devices. build_ata_cmd() figures looks at the devtype argument
and fills in the correct CCB type and CDB or ATA registers.

Revamp the vendor table to remove the previous
vendor-specific ATA entries and use a generic ATA vendor
placeholder. We currently use the same method for all ATA
drives, although we may have to add vendor-specific
behavior once we test this with more drives.

sbin/camcontrol/progress.c:
In progress_draw(), make barlength a signed value so that
we can easily detect a negative value.

If barlength (the length of the progress bar) would wind up
negative due to a small TTY width or a large filename,
set the bar length to the new minimum (10 stars) and
truncate the user's filename. We will truncate it down to
0 characters if necessary.

Calculate a new prefix_len variable (user's filename length)
and use it as the precision when printing the filename.

sbin/camcontrol/camcontrol.c:
Implement a new camcontrol(8) subcommand, "opcodes". The
opcodes subcommand allows displaying the entire list of
SCSI commands supported by a device, or details on an
individual command. In either case, it can display
nominal and recommended timeout values.

Add the scsiopcodes() function, which calls the new
scsigetopcodes() function to fetch opcode data from a
drive.

Add two new functions, scsiprintoneopcode() and
scsiprintopcodes(), which print information about one
opcode or all opcodes, respectively.

Remove the get_disk_type() function. It is no longer used.

Add a new function, dev_has_vpd_page(), that fetches the
supported INQUIRY VPD list from a device and tells the
caller whether the requested VPD page is available.

Add a new function, get_device_type(), that returns a more
precise device type than the old get_disk_type() function.
The get_disk_type() function only distinguished between
SCSI and ATA devices, and SATA devices behind a SCSI to ATA
translation layer were considered to be "SCSI".

get_device_type() offers a third type, CC_DT_ATA_BEHIND_SCSI.
We need to know this to know whether to attempt to send ATA
passthrough commands. If the device has the ATA
Information VPD page (0x89), then it is an ATA device
behind a SCSI to ATA translation layer.

Remove the type argument from the fwdownload() subcommand.

Add a new function, build_ata_cmd(), that will take one set
of common arguments and build either a SCSI or ATA I/O CCB,
depending on the device type passed in.

sbin/camcontrol/camcontrol.h:
Add a prototype for scsigetopcodes().

Add a new enumeration, camcontrol_devtype.

Add prototypes for dev_has_vpd_page(), get_device_type()
and build_ata_cmd().

Remove the type argument from the fwdownload() subcommand.

sbin/camcontrol/camcontrol.8
Explain that the fwdownload subcommand will use the drive
recommended timeout if available, and that the user can
override the timeout.

Document the new opcodes subcommand.

Explain that we will attempt to download firmware to any
SATA device.

Document supported SCSI vendors, and models tested if known.

Explain the commands used to download firmware for the
three different drive and controller combinations.

Document that the -v flag in simulation mode for the fwdownload
subcommand will print out the SCSI CDBs or ATA registers that would
be used.

sys/cam/scsi/scsi_all.h:
Add new bit definitions for the one opcode descriptor for
the REPORT SUPPORTED OPCODES command.

Add a function prototype for scsi_report_supported_opcodes().

sys/cam/scsi/scsi_all.c:
Add a new CDB building function, scsi_report_supported_opcodes().

Sponsored by: Spectra Logic
MFC after: 1 week

show more ...

Fix resource leaks

Found by: Coverity Scan, CID 1016673, 1007118
Approved by: re

Add firmware downloading support for Samsung drives

Tested on Samsung SM1625 SSDs.

Improve firmware download status check. Previous check was insufficient for
ATA since it ignored transport errors like command timeouts, while for SCSI
it was just wrong.

More -Wmissing-variable-declarations fixes.

In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variabl

More -Wmissing-variable-declarations fixes.

In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.

show more ...

Update the 'fwdownload' command to also flash disks connected over an
ATA/SATA transport. The detection logic is automatic, so it should Just
Work. While here, also improve the progress meter that

Update the 'fwdownload' command to also flash disks connected over an
ATA/SATA transport. The detection logic is automatic, so it should Just
Work. While here, also improve the progress meter that is displayed
during firmware download.

Submitted by: Alistair Crooks
Obtained from: Netflix, Inc.
MFC after: 3 days

show more ...

Add missing static keywords to global variables in camcontrol.

While there, make the vendor list const and add appropriate keywords to
functions that use this list.