| Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback |
| Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition | ||
|
|
||
 |
||||||||||||
Solution Type Technical Instruction Sure Solution 1020204.1 : Collecting snapshot on ILOM 3.x and later platforms
PreviouslyPublishedAs 254168
Applies to:Sun Netra T5220 ServerSun Netra T5440 Server Sun Blade T6320 Server Module Sun SPARC Enterprise T5120 Server Sun SPARC Enterprise T5140 Server All Platforms To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community, Oracle Solaris Entrylevel Servers. GoalCollecting snapshot on ILOM 3.x and later platforms.SolutionDescriptionCollecting snapshot on ILOM 3.x and later platforms. Steps to Follow The Snapshot utility provides a single solution to collect SP data for use by Sun Services personnel to diagnose problems. Overview The Snapshot utility provides a single solution to collect SP data for use by Sun[TM] Services personnel to diagnose problems. The utility collects log files, runs various commands and collects their output, and sends the data collection as a zip file to a user defined location. The resulting file is a zip file. It is possible to invoke snapshot in normal mode (via DMTF CLI and BUI), Service mode (via DMTF CLI) and Escalation mode (as a regular bash command). Collecting the snapshot requires the "a" role. Snapshot supports the SFTP (Secure File Transfer Protocal) and FTP (File Transfer Protocal) download protocols as well as HTTPS when using the browser as the target in the BUI. It is also possible to collect the snapshot on a USB stick on platforms that provide this feature. Snapshot also supports encrypting the entire output file. The information to be collected is organized in logsets. The logsets are :
See an example of a the master CONFIG file in attachment (CONFIG). Note : There might be some platform-specific commands added to the master file. Some additional commands (hdtl, spdiag, hostdiags ...) are collected as well for C10 or Galaxy platforms. These commands are added to the master file. See an example of additional commands for Galaxy G12 (CONFIG_galaxy_G12) See an example of additional commands for C10 Vayu (CONFIG_C10_vayu). A copy of the config file for any given platform can be obtained by taking a snapshot from that platform. The logsets are grouped in datasets that are available from CLI/BUI and defined as following :
These versions are intended for situations where the SP/CMM software is malfunctioning. Using this dataset causes snapshot to only collect "log" files as indicated by their entries in the snapshot config file. No commands or normal files are collected. Note : It is possible to check which logsets were used and if the log-only option was used via the README file available in the snapshot directory. Example from a normal-logonly: bash-3.00$ more sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-24-37/README Archive Name: sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-24-37.zip Config File: /usr/local/bin/../lib/snapshot/snapshot.conf Version: 1.1 LOG-ONLY MODE: no Log Sets: ioh SP SW DOWN: no Max Domains: 1 It is possible to encrypt the resulting zip file. The user will be prompted for the password used for encryption in order to decrypt the file. See below for the details. The snapshot utility is implemented as an spsh target in /X/diag namespace with the following properties :
/X/diag/snapshot : Take snapshot of system for diagnostic purposes Targets: Properties: dataset : dataset dataset : Possible values = normal, normal-logonly, fruid, fruid-logonly, full, full-logonly dataset : User role required for set = a dump_uri : initiate snapshot to URI. URI syntax and examples: ftp://user[:password]@host//absolute-directory-path/ ftp://user[:password]@host/relative-directory-path/ sftp://user[:password]@host/absolute-directory-path/ media://media-target ex: sftp://jane@1.2.3.4/tmp/ ex: media://thumbdrive dump_uri : Possible values = sftp, ftp, media dump_uri : User role required for set = a encrypt_output : encrypt snapshot output file encrypt_output : Possible values = true, false encrypt_output : User role required for set = a result : snapshot command result result : User role required for set = a Collect the snapshot Snapshot supports HTTPS, SFTP and FTP download protocols and encrypt file and can be invoked from Normal, Service and Escalation mode. The data collection will start as soon as the dump_uri property is defined. Here are the details of the /X/diag/snapshot target. Define the dataset Before collecting the information, the dataset must be set in order to collect the proper level of information. As previously stated, the logsets are grouped in datasets that are available from CLI/BUI and defined as following :
The dataset is a property of the /X/diag/snapshot target and may be changed prior to starting the data collection. The default value is "normal", which will be sufficient for the vast majority of diagnostic cases. -> show /X/diag/snapshot /X/diag/snapshot Targets: Properties: dataset = normal dump_uri = (Cannot show property) encrypt_output = false result = (none) Commands: cd set show -> help /X/diag/snapshot dataset Properties: dataset : dataset dataset : Possible values = normal, normal-logonly, fruid, fruid-logonly, full, full-logonly dataset : User role required for set = a -> set /X/diag/snapshot dataset=full Set 'dataset' to 'full' -> show /X/diag/snapshot /X/diag/snapshot Targets: Properties: dataset = full dump_uri = (Cannot show property) encrypt_output = false result = (none) Commands: cd set show Define the encryption mode It is possible to encrypt the resulting zip file. The user will be prompted for an encryption password at time of starting the collection. The user will be prompted for the password used for encryption in order to decrypt the file. Example : -> show /X/diag/snapshot encrypt_output /X/diag/snapshot Properties: encrypt_output = false -> set /X/diag/snapshot encrypt_output=true Set 'encrypt_output' to 'true' -> show /X/diag/snapshot encrypt_output /X/diag/snapshot Properties: encrypt_output = true -> set /X/diag/snapshot dump_uri=sftp://user@xx.xx.xx.xx/tmp/Tests Enter remote user password: ********* Enter encryption passphrase for snapshot output file: *** Confirm encryption passphrase for snapshot output file: *** Set 'dump_uri' to 'sftp://user@xx.xx.xx.xx/tmp/Tests' When the zip.e file is ready, then decrypt and unzip. % openssl aes-128-cbc -d -in sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-54-15.zip.e -out sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-54-15.zip enter aes-128-cbc decryption password: ******* % unzip -q sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-54-15.zip Start the data collection The data collection will start as soon as the dump_uri property is properly set. It is possible to send the zip file resulting from the snapshot data collection to a remote system via https (BUI), SFTP (BUI/CLI) and FTP (BUI/CLI). If the platform supports this, it is also possible to store the zip file on a USB stick local to the SP/CMM (BUI/CLI). The dump_uri property will contain this information. -> help /X/diag/snapshot dump_uri Properties: dump_uri : initiate snapshot to URI dump_uri : Possible values = sftp, ftp, media dump_uri : User role required for set = a Example using the FTP protocol : -> set dump_uri=ftp://user@xx.xx.xx.xx//tmp/ Enter remote user password: ********* Set 'dump_uri' to 'ftp://user@xx.xx.xx.xx//tmp/' Notes :
Example using the SFTP protocol : -> set /X/diag/snapshot dump_uri=sftp://user@xx.xx.xx.xx/home/user Enter remote user password: ********* Set 'dump_uri' to 'sftp://user@xx.xx.xx.xx/home/user' If the URI contains the password then the system will not prompt you for it. -> set /X/diag/snapshot dump_uri=sftp://user:password@xx.xx.xx.xx/home/user Set 'dump_uri' to 'sftp://user@xx.xx.xx.xx/home/user' In order to collect the snapshot on a USB stick the appropriate target must be specified. These are the targets defined in the /X/media namespace. Example : -> show /X/media /X/media Targets: thumbdrive usb0 usb1 Properties: present = thumbdrive:Thumbdrive Commands: cd show Using the appropriate target to set the dump_uri will start the collection. The "present" property is used by the BUI to build its drop-down list of available devices. Its contents are: target:Label[;target:Label]... Examples -> cd /X/diag/snapshot/ /X/diag/snapshot -> set dump_uri=media://thumbdrive or -> set dump_uri=media://thumbdrive/my/directory or -> set dump_uri=media://usb0/my/directory If no directory is specified after the media target, snapshot will store the output data in the /snapshot_data directory in the root of the USB device. This is due to a limitation in the number of files and directories allowed in the root directory of the FAT filesystem on USB devices. If a directory is specified as shown above, then that directory is used. Note that alternate directories on USB devices can only be specified via the CLI. Service mode (logsets = S) When the Service mode is enabled, it is possible to collect the snapshot including the extra information gathered while running in Service mode. This extra information will be stored in the spos_info/service/, spos_logs/service/ and ilom/service/ directories. Similar to Normal mode, set the dataset to the appropriate value and start the data collection by setting the dump_uri property. Setting the dataset to normal will result in using the iohS logsets. Setting the dataset to fruid will result in using the FiohS logsets. Setting the dataset to full will result in using the FiohdS logsets. -> show SESSION mode /X/sessions/22 Properties: mode = service -> show /X/diag/snapshot dataset /X/diag/snapshot Properties: dataset = fruid -> set /X/diag/snapshot dump_uri=sftp://user@xx.xx.xx.xx/home/user Enter remote user password: ********* Set 'dump_uri' to 'sftp://user@xx.xx.xx.xx/home/user' Escalation mode (logset = E) When running in Escalation mode, it is possible to invoke snapshot from the bash command line. The dataset/logset can be defined via the "-L" option. WARNING: No user role checking is performed in escalation mode. BE CAREFUL especially with the diagnostics logset. This extra information will be stored in the ilom/escalation/ and spos_info/escalation/ directories. Example : bash-2.05b# snapshot Usage: snapshot [-l] [-v] [-q] [-{y|n}] [-e [-P encryption-password]] [-L ] [-p user-password] -u set = one or more letters from logset field in configuration file entries destination-URI (i.e. the target directory) may be specified as: file:///path media:///path media://thumbdrive/path protocol://host/path protocol://username@host/path protocol://username:password@host/path protocol = 'sftp', 'tftp', 'ftp', 'ftps', 'http', or 'https' mediadevice: e.g. "usb0". See mediaadm --enumerate bash-2.05b# snapshot -L E -u sftp://user@xx.xx.xx.xx/tmp/Tests Enter password for user "user": Collecting data into sftp://user@xx.xx.xx.xx/tmp/Tests/sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T21-12-24.zip Snapshot Complete. bash-2.05b# snapshot -L oihFE -u sftp://user@xx.xx.xx.xx/tmp/Tests Enter password for user "user": Collecting data into sftp://user@xx.xx.xx.xx/tmp/Tests/sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T21-15-28.zip Snapshot Complete. Collect from the Browser User Interface It is possible to collect data and invoke snapshot from the BUI. The BUI offers to select the dataset to be used or to specify the logset; as well as the -logonly options. It is also possible to select the protocol or the local service (usb, thumbdrive) to be used. Note : The Service and Escalation datasets are not available from the BUI. See the attached screenshot. Check the result The /X/diag/snapshot target has a "result" property that reports information about the status of the data collection. When the snapshot is running. -> show /X/diag/snapshot result /X/diag/snapshot Properties: result = Running When the snapshot has completed successfully. -> show /X/diag/snapshot result /X/diag/snapshot Properties: result = Collecting data into sftp://user@xx.xx.xx.xx/home/user/sn-br-sp-sca11_xx.xx.xx.xx_2009-01-23T09-33-07.zip Snapshot Complete. Done. When a problem occurred while collecting the data, the result property will also return the reason for the failure. -> ls /X/diag/snapshot Targets: Properties: dataset = normal dump_uri = (Cannot show property) encrypt_output = false result = Access denied to remote resource Exited with error code 109 Commands: cd set show or -> ls /X/diag/snapshot Targets: Properties: dataset = normal dump_uri = (Cannot show property) encrypt_output = false result = Error: Check target_URI protocol and syntax Commands: cd set show Structure of the snapshot directory After unzipping the resulting snapshot zip file, the following structure is available :
See an example of a the master CONFIG file in attachment (CONFIG). Note : There might be some platform-specific commands added to the master file. Some additional commands (hdtl, spdiag, hostdiags ...) are collected as well for C10 or Galaxy platforms. These commands are added to the master file. See an example of additional commands for Galaxy G12 (CONFIG_galaxy_G12) See an example of additional commands for C10 Vayu (CONFIG_C10_vayu). A copy of the config file for any given platform can be obtained by taking a snapshot from that platform. As an example : bash-3.00$ pwd sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-44-34 bash-3.00$ more README Archive Name: sn-br-sp-sca11_xx.xx.xx.xx_2009-03-02T15-44-34.zip Config File: /usr/local/bin/../lib/snapshot/snapshot.conf Version: 1.1 LOG-ONLY MODE: no Log Sets: Fiohd SP SW DOWN: no Max Domains: 1 bash-3.00$ ls -la total 194 drwx--x--x 7 sdutille divers 9 Mar 2 15:47 . drwxrwxrwx 159 sdutille staff 456 Mar 2 22:47 .. -rw------- 1 sdutille divers 6090 Dec 18 06:51 CONFIG -rw------- 1 sdutille divers 200 Mar 2 07:44 README drwx--x--x 2 sdutille divers 38 Mar 2 15:47 fruid drwx--x--x 3 sdutille divers 8 Mar 2 15:47 ilom drwx--x--x 2 sdutille divers 6 Mar 2 15:47 ipmi drwx--x--x 3 sdutille divers 17 Mar 2 15:47 spos_info drwx--x--x 2 sdutille divers 11 Mar 2 15:47 spos_logs Making ILOM snapshot work with explorer: Explorer 6.1 will be able to collect snapshot data from systems running ILOM 3.0. Background: It is assumed that explorer will be run as root on a host dedicated to collecting one or more ILOM snapshot output files (the Explorer Host). This is potentially the platform host itself. In general, explorer will login to the SP or CMM using SSH public key authentication, start snapshot with its output directed back to the host running explorer, then collect the .zip output file once snapshot is complete. The zip file will be collected at the end of the explorer execution and made available in the ../ilom directory of the explorer. There are several preparation steps necessary for an ILOM SP/CMM to work with explorer. Setup public key authentication on SP/CMM 1. Create a user on the SP/CMM with the administration (a) role. Below, replace "X" with "SP" or "CMM" based on your platform. -> create /X/users/expluser Creating user... Enter new password: create: Password length must be between 8 and 16 characters Enter new password: ********* Enter new password again: ********* Created /X/users/expluser -> set /X/users/expluser role=ao Set 'role' to 'ao' 2. Load the public key. So that the Explorer Host root user can login to the SP or CMM as user expluser, you must make the Explorer Host's root user's public key available to the SP or CMM via one of its supported protocols. The load_uri supports numerous protocols. To use sftp, scp or ftp, be sure to supply the username and password in the URI. e.g. set load_uri=sftp://username:password@host/absolute/path/to/public/key First, generate a key on the Explorer Host. This is the key that will be loaded on the SP / CMM (do not use passphrase) : bash-3.00# ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (//.ssh/id_dsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in //.ssh/id_dsa. Your public key has been saved in //.ssh/id_dsa.pub. The key fingerprint is: d9:93:ce:38:53:32:65:9c:60:30:be:00:76:df:05:fe root@host Log to the SP/CMM from the Explorer Host as expluser : bash-3.00# ssh expluser@xx.xx.xx.xx Password: Waiting for daemons to initialize... Daemons ready Sun(TM) Integrated Lights Out Manager Version 3.0.4.0-bld_55-t Copyright 2010 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. -> Make sure to clear any existing key : -> cd /X/users/expluser/ssh/keys/1 /X/users/expluser/ssh/keys/1 -> set clear_action=true Are you sure you want to clear /X/users/expluser/ssh/keys/1 (y/n)? y Set 'clear_action' to 'true' Then load the key on the SP : -> cd /X/users/expluser/ssh/keys/1 /X/users/expluser/ssh/keys/1 -> set load_uri=tftp://1.2.3.4/.ssh/id_dsa.pub Load successful. -> ls -d properties /X/users/expluser/ssh/keys/1 Properties: fingerprint = c0:98:22:33:60:84:ec:b8:88:ba:cb:5c:fc:1c:6b:37 algorithm = ssh-dss embedded_comment = (none) bit_length = 1024 load_uri = (Cannot show property) clear_action = (Cannot show property) Note that using the tfpt protocol is just an example here. The load_uri supports numerous protocols. To use sftp, scp or ftp, be sure to supply the username and password in the URI. e.g. set load_uri=scp://root:password@host/.ssh/id_dsa.pub Refer to the Oracle Integrated Lights Out Manager (ILOM) 3.0 CLI Procedures Guide for the syntax and usage. 3. Log again to the SP user account from the Explorer Host From the account on the Explorer Host running explorer (usually root), login once to the user account just created on the SP in order to accept the host key # ssh expluser@xx.xx.xx.xx The authenticity of host 'xx.xx.xx.xx' can't be established. RSA key fingerprint is ec:29:8c:8c:3d:82:59:15:f3:4b:fe:dd:12:52:7e:49. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'xx.xx.xx.xx' (RSA) to the list of known hosts. Sun(TM) Integrated Lights Out Manager Version 3.0.4.0-bld_55-t Copyright 2008 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. -> Note that no password is now required. During the explorer execution, if the snapshot collection fails with "ilomsnapshot_start: SSH hostkey must be accepted" , see example: # /opt/SUNWexplo/bin/explorer : 17:27:21 T5440[29340] explorer: explorer ID: explorer.84aabf4a.T5440-2010.04.22.08.27 17:27:22 T5440[29340] ilomsnapshot_start: RUNNING 17:27:22 T5440[29340] ilomsnapshot_start: SSH hostkey must be accepted. : then generate a new key on the SP : - Log in to the SP - Set the key type by typing the following: -> set /X/services/ssh generate_new_key_type=dsa|rsa - Set the action to true. -> set /X/services/ssh generate_new_key_action=true When running explorer, you can check the result for the snapshot data collection via the ilom/snapshot_*.* files. If the "ilom/snapshot_start.err" file reports a "Host key verification failed." Example: # cat /opt/SUNWexplo/output/explorer.xxxxxxxx.T5440-2010.04.26.07.10/ilom/snapshot_start.err *********************************************************** * WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! * *********************************************************** IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the RSA host key has just been changed. The fingerprint for the RSA key sent by the remote host is db:83:8e:44:xx:yy:zz:7d:f5:89:ef:88:c6:aa:bb:cc. Please contact your system administrator. Add correct host key in /.ssh/known_hosts to get rid of this message. Offending key in /.ssh/known_hosts:2 RSA host key for 10.mm.mmm.YYY has changed and you have requested strict checking. Host key verification failed. Then the old key for the SP must be removed from the Explorer Host : # rm /.ssh/known_hosts Setup a user on the Explorer Host to receive snapshot data. There must be a user account on the Explorer Host that can be accessed from the SP or CMM via ftp or sftp. This user account must be able to write into the dropoff directory. 1. Create the user # useradd expltest # passwd expltest New Password: Re-enter new Password: passwd: password successfully changed for expltest See svcadm(1M) if necessary to allow ftp or sftp 2. Configure ilomsnapshotinput.txt The ilomsnapshotinput.txt file lives in /etc/opt/SUNWexplo on the Explorer Host. This file contains one line per SP or CMM to have its ILOM snapshot data collected. Note that explorer requires the permissions for ilomsnapshotinput.txt to be 400 or 600. Here is the empty file that is distributed with Explorer: # Input file for extended ilom snapshot data collection # Format: # SPHOST SPUSER PROTO DESTHOST DESTPICKUP DESTDROPOFF DESTUSER DESTPASSWORD # Explorer requires the mode of ilomsnapshotinput.txt is 0400 or 0600 # SPHOST: hostname or IP address of SP or CMM. # SPUSER: explorer logs in to SP/CMM as this user. # PROTO={ftp|sftp} # DESTHOST: hostname or IP address of system running explorer (this system). # Use "-" to have explorer lookup IP address for hostname from hosts table. # DESTPICKUP: Absolute path of directory on this system where explorer will be # receive snapshots. Use "-" to use /tmp. # DESTDROPOFF: directory where snapshot will deposit output. When PROTO is sftp # this must be an absolute path to a directory. Use "-" to use /tmp. # DESTUSER: snapshot logs in to the system running explorer as this user # DESTPASSWORD: DESTUSER's password # # mysp.mydom spuser sftp ftpzone-lomnet /zones/ftpzone/export/ftp/incoming /ftp/incoming explrecvuser explrecvuser-password The file allows for the user explrecvuser to have a different view of the filesystem than the Explorer Host root user. This may be useful in systems with zones or other configurations that want to provide increased security around the user explrecvuser. In most cases, the default configuration will work fine. Add an entry in ilomsnapshotinput.txt for every SP/CMM that will have its snapshot data collected by this Explorer Host. For example: * xx.xx.xx.XX expluser sftp - - - expltest expltest * The user on the explorer host (expltest in this example) can have its login access restricted or removed and the account can be made very secure, just so long as it is possible to write into the dropoff directory on the explorer host using ftp or sftp. Starting from Explorer 6.4 and later, it's possible to specify the dataset (normal, full, fruid) in the ilomsnapshotinput.txt file. Example : # Input file for extended ilom snapshot data collection # DESTPASSWORD:DESTUSER's password # mysp.mydom spuser sftp ftpzone-lomnet /zones/ftpzone/export/ftp/incoming /ftp/incoming fruid explrecvuser explrecvuser-password 3. Invoke Explorer Run Explorer as normal. # explorer Feb 07 18:23:35 explorer_host[5037] explorer: explorer ID: explorer.xxxxxxxx.explorer_host-2011.02.07.10.23 Run Explorer only to collect ILOM snapshot. # explorer -w !default,ilomsnapshot 4. When the explorer data collection has completed, the snapshot will then be available in the 'ilom' directory. Example : # pwd Sun SPARC Enterprise T5220 Server Sun Blade T6320 Server Module Netra T5220 AC Sun Netra T5220 Server Sun SPARC Enterprise T5140 Server Sun SPARC Enterprise T5240 Server Sun Netra T5440 Server Sun SPARC Enterprise T5440 Server Internal Comments For internal Sun use only. ILOM, snapshot, explorer Attachments This solution has no attachment |
||||||||||||
|
||||||||||||