2016-10-27 Maxim Nestratov vz: remove Bridged network name and rename Routed It's funny, but Routed network name was incorrect. We should use host-routed instead. vz: support type=bridge network interface type correctly Recently, libprlsdk got a separate flag PNA_BRIDGE corresponding to type=bridge libvirt network interfaces. Let's use it and get rid of all workarounds previously added to support it. 2016-10-27 Chen Hanxiao qemu: Forbid pinning vCPUs for TCG domain We don't support cpu pinning for TCG domains because QEMU runs them in one thread only. But vcpupin command was able to set them, which resulted in a failed startup, so make sure that doesn't happen. 2016-10-27 Ján Tomko Recreate the USB address cache at reconnect When starting a new domain, we allocate the USB addresses and keep an address cache in the domain object's private data. However this data is lost on libvirtd restart. Also generate the address cache if all the addresses have been specified, so that devices hotplugged after libvirtd restart also get theirs assigned. https://bugzilla.redhat.com/show_bug.cgi?id=1387666 2016-10-27 Ján Tomko Do not try to release virtio serial addresses Return 0 instead of 1, so that qemuDomainAttachChrDevice does not assume the address neeeds to be released on error. No functional change, since qemuDomainReleaseDeviceAddress has been a noop for virtio serial addresses since the address cache was removed in commit 19a148b. 2016-10-27 Ján Tomko Fix crash on usb-serial hotplug For domains with no USB address cache, we should not attempt to generate a USB address. https://bugzilla.redhat.com/show_bug.cgi?id=1387665 2016-10-27 Ján Tomko Return directly from qemuDomainAttachChrDeviceAssignAddr This function should never need a cleanup section. 2016-10-27 Ján Tomko Introduce virDomainVirtioSerialAddrAutoAssign again This time do not require an address cache as a parameter. Simplify qemuDomainAttachChrDeviceAssignAddr to not generate the virtio serial address cache for devices of other types. Partially reverts commit 925fa4b. 2016-10-27 Ján Tomko Add 'FromCache' to virDomainVirtioSerialAddrAutoAssign Commit 19a148b dropped the cache from QEMU's private domain object. Assume the callers do not have the cache by default and use a longer name for the internal ones that do. This makes the shorter 'virDomainVirtioSerialAddrAutoAssign' name availabe for a function that will not require the cache. 2016-10-27 Martin Kletzander virsh: Fix on_shutdown to on_poweroff The 'on_shutdown' was the name in Xen, there was never such option in libvirt's domain XML. Reported-by: Ruben Kerkhof 2016-10-26 Sławek Kapłoński qemu, lxc: Raise error message when resuming running domain When user tries to resume already running domain (Qemu or LXC) VIR_ERR_OPERATION_INVALID error should be raised with message that domain is already running. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1009008 2016-10-26 Andrea Bolognani wireshark: Rename plugindir to ws_plugindir Since we're using autoconf to substitute the right value in Makefile.am now, we can use a less generic name without running into circular dependencies. 2016-10-26 Andrea Bolognani wireshark: Inject $(prefix) at the right time Adding $(prefix) in Makefile.am, as we were doing, means that it would be prepended even when using --with-ws-plugindir, which is something we don't want to happen. Instead, we add it beforehand but take care that it doesn't get expanded until make is called. 2016-10-26 Andrea Bolognani wireshark: Strip prefix correctly Even when we're building $plugindir ourselves because we can't retrieve it using pkg-config, we still want to strip the prefix, except in that case it would be the same prefix we're using for building libvirt. The fact that $plugindir is missing also doesn't tell us anything about $ws_prefix, so we have to handle the two variables separately. 2016-10-26 Andrea Bolognani wireshark: Hoist $ws_prefix declaration Keep all variable declarations close together. wireshark: Introduce $ws_modversion Use a separate variable instead of setting it inline for slightly cleaner code. 2016-10-26 Gema Gomez qemu: Add support for using AES secret for SCSI hotplug Support for virtio disks was added in commit id 'fceeeda', but not for SCSI drives. Add the secret for the server when hotplugging a SCSI drive. No need to make any adjustments for unplug since that's handled during the qemuDomainDetachDiskDevice call to qemuDomainRemoveDiskDevice in the qemuDomainDetachDeviceDiskLive switch. Added a test to/for the command line processing to show the command line options when adding a SCSI drive for the guest. 2016-10-26 John Ferlan qemu: Add secret object hotplug for TCP chardev TLS https://bugzilla.redhat.com/show_bug.cgi?id=1300776 Complete the implementation of support for TLS encryption on chardev TCP transports by adding the hotplug ability of a secret to generate the passwordid for the TLS object for chrdev, RNG, and redirdev. Fix up the order of object removal on failure to be the inverse of the attempted attach (for redirdev, chr, rng) - for each the tls object was being removed before the chardev backend. Likewise, add the ability to hot unplug that secret object as well and be sure the order of unplug matches that inverse order of plug. 2016-10-26 John Ferlan qemu: Add a secret object to/for a char source dev Add the secret object so the 'passwordid=' can be added if the command line if there's a secret defined in/on the host for TCP chardev TLS objects. Preparation for the secret involves adding the secinfo to the char source device prior to command line processing. There are multiple possibilities for TCP chardev source backend usage. Add test for at least a serial chardev as an example. 2016-10-26 John Ferlan qemu: Need to remove TLS object in RemoveRNGDevice Commit id '6e6b4bfc' added the object, but forgot the other end. qemu: Fix depedency order in qemuRemoveDiskDevice Need to remove the drive first, then the secobj and/or encobj if they exist. This is because the drive has a dependency on secobj (or the secret for the networked storage server) and/or the encobj (or the secret for the LUKS encrypted volume). Deleting either object first leaves an drive without it's respective objects. 2016-10-25 John Ferlan virsh: Add _length parameters to virsh output https://bugzilla.redhat.com/show_bug.cgi?id=1349898 Add the duration parameters to the virsh input/output for blkdeviotune command and describe them in the pod file. 2016-10-25 John Ferlan virsh: Create macros to add IOTUNE values Rework the repetitive lines to add iotune values into easier to read macros. One to handle the SCALED values and one to handle the non scaled values. 2016-10-25 John Ferlan qemu: Add the length options to the iotune command line Add in the block I/O throttling length/duration parameter to the command line if supported. If not supported, fail command creation. Add the xml2argvtest for testing. 2016-10-25 John Ferlan conf: Add support for blkiotune "_length" options Modify _virDomainBlockIoTuneInfo and rng schema to support the _length options for bps/iops throttling values. Document the new values. 2016-10-25 John Ferlan qemu: Add length for bps/iops throttling parameters to driver Add support for a duration/length for the bps/iops and friends. Modify the API in order to add the "blkdeviotune." specific definitions for the iotune throttling duration/length options total_bytes_sec_max_length write_bytes_sec_max_length read_bytes_sec_max_length total_iops_sec_max_length write_iops_sec_max_length read_iops_sec_max_length 2016-10-25 John Ferlan caps: Add new capability for the bps/iops throttling length Add the capability to detect if the qemu binary can support the feature to use bps-max-length and friends. 2016-10-25 John Ferlan include: Add new definitions for duration for bps/iops throttling Add new options to allow proving a duration/length in seconds to allow the bps/iops (and friends) to occur: total_bytes_sec_max_length write_bytes_sec_max_length read_bytes_sec_max_length total_iops_sec_max_length write_iops_sec_max_length read_iops_sec_max_length Add continue for compiler hint to return to for control 2016-10-25 John Ferlan qemu: Introduce qemuDomainSetBlockIoTuneDefaults Create a helper to set the bytes/iops iotune default values based on the current qemu setting for both the live and persistent definitions. NB: This also fixes an unreported bug where the persistent values for *_max and size_iops_sec would be set back to 0 if unrelated persistent values were set. qemu: Move setting of conf_disk in qemuDomainSetBlockIoTune Since persistent_def is the only place that uses it, let's just keep it closer to where it's used. 2016-10-25 John Ferlan qemu: Return real error message for block_set_io_throttle This patch will also adjust the qemuMonitorJSONSetBlockIoThrottle error procession so that rather than returning/displaying: "error: internal error: Unexpected error" Fetch the actual error message from qemu and display that 2016-10-25 John Ferlan qemu: Create a macro to handle setting bytes/iops iotune values Create a macros to hide all the comparisons for each of the fields. Add a 'continue;' for a compiler hint that we only need to find one this should be similar enough to the if - elseif - elseif logic. 2016-10-25 John Ferlan qemu: Move TLS object remove from DetachChr to RemoveChr Commit id '2c32237' added the TLS object removal to the DetachChrDevice all when it should have been added to the RemoveChrDevice since that's the norm for similar processing (e.g. disk) 2016-10-25 Ján Tomko virQEMUCapsReset: also clear out hostCPUModel After succesfully reading an outdated caps cache from disk, calling virQEMUCapsReset did not properly clear out the calculated host CPU model. This lead to a memory leak when the host CPU model pointer was overwritten later in virQEMUCapsNewForBinaryInternal. Introduced by commit 68c70118. 2016-10-25 Jim Fehlig libxl: fix leaking of allocated migration ports Although the migration port is immediately released in the finish phase of migration, it was never set in the domain private object when allocated in the prepare phase. So libxlDomainMigrationFinish() always released a 0-initialized migrationPort, leaking any allocated port. After enough migrations to exhaust the migration port pool, migration would fail with error: internal error: Unable to find an unused port in range 'migration' (49152-49216) Fix it by setting libxlDomainObjPrivate->migrationPort to the port allocated in the prepare phase. While at it, also fix leaking an allocated port if the prepare phase fails. 2016-10-25 Viktor Mihajlovski doc: update virsh domstats documentation for vcpu statistics Added description for new vcpu..halted statistics value. While there, also added a description for vcpu..wait and clarified the units displayed for time and wait. Reviewed-by: Bjoern Walk 2016-10-25 Viktor Mihajlovski qemu: add vcpu.n.halted to vcpu domain stats Extended qemuDomainGetStatsVcpu to include the per vcpu halted indicator if reported by QEMU. The key for new boolean value has the format "vcpu..halted". Reviewed-by: Bjoern Walk 2016-10-25 Viktor Mihajlovski qemu: Add domain support for VCPU halted state Adding a field to the domain's private vcpu object to hold the halted state information. Adding two functions in support of the halted state: - qemuDomainGetVcpuHalted: retrieve the halted state from a private vcpu object - qemuDomainRefreshVcpuHalted: obtain the per-vcpu halted states via qemu monitor and store the results in the private vcpu objects Reviewed-by: Bjoern Walk Reviewed-by: Hao QingFeng 2016-10-25 Viktor Mihajlovski qemu: Add monitor support for CPU halted state Extended the qemuMonitorCPUInfo with a halted flag. Extract the halted flag for both text and JSON monitor. 2016-10-24 Laine Stump qemu: replace calls to virDomainPCIAddressReserveNext*() with static function An upcoming commit will remove the "flag" argument from all the calls to reserve the next available address|slot, but I don't want to change the arguments in the hypervisor-agnostic virDomainPCIAddressReserveNext*() functions, so this patch places a simple qemu-specific wrapper around those functions - the new functions don't take a flags arg, but grab it from the device's info->pciConnectFlags. conf: make virDomainPCIAddressGetNextSlot() a local static function This function is no longer needed outside of domain_addr.c. qemu: use virDomainPCIAddressReserveNextAddr in qemuDomainAssignDevicePCISlots instead of calling virDomainPCIAddressGetNextSlot() (which I want to turn into a local static in domain_addr.c). 2016-10-24 Laine Stump conf: new function virDomainPCIAddressReserveNextAddr() There is an existing virDomainPCIAddressReserveNextSlot() which will reserve all functions of the next available PCI slot. One place in the qemu PCI address assignment code requires reserving a *single* function of the next available PCI slot. This patch modifies and renames virDomainPCIAddressReserveNextSlot() so that it can fulfill both the original purpose and the need to reserve a single function. (This is being done so that the abovementioned code in qemu can have its "kind of open coded" solution replaced with a call to this new function). 2016-10-24 Pavel Hrdina domain: fix migration to older libvirt Since TLS was introduced hostwide for libvirt 2.3.0 and a domain configurable haveTLS was implemented for libvirt 2.4.0, we have to modify the migratable XML for specific case where the 'tls' attribute is based on setting from qemu.conf. The "tlsFromConfig" is libvirt internal attribute and is stored only in status XML to ensure that when libvirtd is restarted this internal flag is not lost by the restart. That flag is used to decide whether we should put *tls* attribute to migratable XML or not. 2016-10-24 Pavel Hrdina domain: Add optional 'tls' attribute for TCP chardev Add an optional "tls='yes|no'" attribute for a TCP chardev. For QEMU, this will allow for disabling the host config setting of the 'chardev_tls' for a domain chardev channel by setting the value to "no" or to attempt to use a host TLS environment when setting the value to "yes" when the host config 'chardev_tls' setting is disabled, but a TLS environment is configured via either the host config 'chardev_tls_x509_cert_dir' or 'default_tls_x509_cert_dir' 2016-10-24 Pavel Hrdina domain_conf: remove union for one member from redirdev struct Currently the union has only one member so remove that union. If there is a need to add a new type of source for new bus in the future this will force the author to add a union and properly check bus type before any access to union member. domain_conf: fix memory leak in virDomainDefAddConsoleCompat 2016-10-24 John Ferlan qemu: Add TLS hotplug for qemuDomainAttachRNGDevice Commit id '2c322378' missed the nuance that the rng backend could be using a TCP chardev and if TLS is enabled on the host, thus will need to have the TLS object added. qemu: Add TLS hotplug for qemuDomainAttachRedirdevDevice Commit id '2c322378' missed the nuance that the redirdev backend could be using a TCP chardev and if TLS is enabled on the host, thus will need to have the TLS object added. qemu: Clean up error path in qemuDomainAttachRedirdevDevice It's about to get more complicated - let's alter the logic to handle various failures. Adds saving of the error as well. qemu: Introduce qemuDomainGetChardevTLSObjects for hotplug As it turns out more than one place will need these objects, so rather than cut-copy-paste in each, make a helper conf: Use virDomainChrSourceDefNew for virDomainRNGDef allocation Rather than VIR_ALLOC() the data, use virDomainChrSourceDefNew in order to get the private data if necessary. conf: Use virDomainChrSourceDefPtr for _virDomainRedirdevDef 'source.chr' Use a pointer and the virDomainChrSourceDefNew() function in order to allocate the structure for _virDomainRedirdevDef. conf: Use virDomainChrSourceDefPtr for _virDomainSmartcardDef 'passthru' Use a pointer and the virDomainChrSourceDefNew() function in order to allocate the structure for _virDomainSmartcardDef. 2016-10-24 Michal Privoznik xenFormatXLDisk: Unify commas pattern for arguments instead of: virBufferAdd(buf, "arg1,"); virBufferAdd(buf, "arg2"); lets have: virBufferAdd(buf, "arg1"); virBufferAdd(buf, ",arg2"); Because it's better. Consider we want to add conditionally arg3. With this change, it's simple: if (cond) virBufferAdd(buf, ",arg3"); with current code there might be a comma hanging at EOL. 2016-10-23 Laine Stump qemu: change first arg of qemuDomainAttachChrDeviceAssignAddr() from virDomainDefPtr to virDomainObjPtr so that the function has access to the other parts of the virDomainObjPtr. Take advantage of this by removing the "priv" arg and retrieving it from the virDomainObjPtr instead. No functional change. 2016-10-23 Laine Stump qemu: make error message in qemuDomainPCIAddressSetCreate more clear. This error should only ever be seen by a developer anyway, but the existing message made even less sense that this new version. qemu: remove superfluous setting of addrs->nbuses This is already set by virDomainPCIAddressSetAlloc(). conf: add typedef for anonymous enum used for memballoon device model For some reason the values of memballoon model are set using an anonymous enum, making it impossible to perform nice tricks like demanding there are cases for all possible values in a switch. This patch turns the anonymous enum into virDomainMemballoonModel. qemu: replace "def->nets[i]" with "net" and "def->sounds[i]" with "sound" More occurences of repeatedly dereferencing the same pointer stored in an array are replaced with the definition of a temporary pointer that is then used directly. No functional change. qemu: replace a lot of "def->controllers[i]" with equivalent "cont" There's no functional change here. This pointer was just used so many times that the extra long lines became annoying. 2016-10-21 John Ferlan conf: Move the privateData from virDomainChrDef to virDomainChrSourceDef Commit id '5f2a132786' should have placed the data in the host source def structure since that's also used by smartcard, redirdev, and rng in order to provide a backend tcp channel. The data in the private structure will be necessary in order to provide the secret properly. This also renames the previous names from "Chardev" to "ChrSource" for the private data structures and API's 2016-10-21 John Ferlan Introduce virDomainChrSourceDefNew for virDomainChrDefPtr Change the virDomainChrDef to use a pointer to 'source' and allocate that pointer during virDomainChrDefNew. This has tremendous "fallout" in the rest of the code which mainly has to change source.$field to source->$field. 2016-10-21 John Ferlan vmx: Use the allocator virDomainChrDefNew Rather than VIR_ALLOC of the virDomainChrDefPtr 2016-10-21 Viktor Mihajlovski doc: Describe the VCPU states returned by virsh vcpuinfo Added a brief description of the VCPU states. 2016-10-21 Ján Tomko qemuBuildHostNetStr: remove dead code This function is never called for VIR_DOMAIN_NET_TYPE_HOSTDEV, and the dead code comment agrees. Introduced by commit 1dcbef8a. 2016-10-21 Ján Tomko qemuBuildHostNetStr: do not start options with a comma Put the comma at the end and trim it later for consistency. 2016-10-21 Ján Tomko qemuBuildHostNetStr: use type_sep earlier When hotplugging networks with ancient QEMUs not supporting QEMU_CAPS_NETDEV, we use space instead of a comma as the separator between the network type and other options. Except for "user", all the network types pass other options and use up the first separator by the time we get to the section that adds the alias (or vlan for QEMUs without CAPS_NETDEV). Since the alias/vlan is mandatory, convert all preceding code to add the separator at the end, removing the need to rewrite type_sep for all types but NET_TYPE_USER. 2016-10-21 Nikolay Shirokovskiy vz: set something in disk driver name Absent driver name attribute is invalid xml. Which in turn makes unusable 'virsh edit' for example. The value does not make much sense and ignored on input so nobody will hurt. 2016-10-21 Nikolay Shirokovskiy vz: add serial number to disk devices vz sdk supports setting serial number only for disk devices. Getting serial upon cdrom(for example) is error however setting is just ignored. Let's check for disk device explicitly for clarity in both cases. Setting serial number for other devices is ignored with an info note just as before. We need usual conversion from "" to NULL in direction vz sdk -> libvirt, because "" is not valid for libvirt and "" means unspecifiend in vz sdk which is NULL for libvirt. 2016-10-21 John Ferlan bhyve: Need to add parameter to virDomainChrDefNew Commit id '5f2a13278' missed this one. 2016-10-21 Mikhail Feoktistov vz: set localhost as vnc address We should set localhost as vnc address in case of empty string. Because Virtuozzo sets 0.0.0.0 as default vnc address. 2016-10-21 Michal Privoznik virt-wireshark.m4: Defer $(prefix) substitution The autoconf manual says we shouldn't be using this variable (and some others) outside of Makefiles as it prevents users providing their own prefix at the installation phase. https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Installation-Directory-Variables.html 2016-10-20 Sławek Kapłoński networkValidate: Forbid new-line char in network name New line character in name of network is now forbidden because it mess virsh output and can be confusing for users. Validation of name is done in network driver, after parsing XML to avoid problems with disappeared network which was already created with new-line char in name. Closes-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=818064 2016-10-20 Sławek Kapłoński network: Use new util function to check name New util function virXMLCheckIllegalChars is now used to test if parsed network contains illegal char '/' in it's name. virxml: Add function to check if string contains some illegal chars This new function can be used to check if e.g. name of XML node don't contains forbidden chars like "/" or "\n". 2016-10-19 John Ferlan qemu: Introduce qemuDomainChardevPrivatePtr Modeled after the qemuDomainHostdevPrivatePtr (commit id '27726d8c'), create a privateData pointer in the _virDomainChardevDef to allow storage of private data for a hypervisor in order to at least temporarily store secret data for usage during qemuBuildCommandLine. NB: Since the qemu_parse_command (qemuParseCommandLine) code is not expecting to restore the secret data, there's no need to add code code to handle this new structure there. 2016-10-19 John Ferlan conf: Introduce {default|chardev}_tls_x509_secret_uuid Add a new qemu.conf variables to store the UUID for the secret that could be used to present credentials to access the TLS chardev. Since this will be a server level and it's possible to use some sort of default, introduce both the default and chardev logic at the same time making the setting of the chardev check for it's own value, then if not present checking whether the default value had been set. 2016-10-19 Cédric Bosdonnat xl: fix 8126d870 broken test xlconfigtest expects the comma, don't clean it up even if there is no target to write. 2016-10-19 Andrea Bolognani spec: Drop support for Fedora < 23 We only claim support for OSs that are still supported by the respective vendors, which means anything older than Fedora 23 is out. Reword the comment a bit to highlight the criteria. 2016-10-19 Cédric Bosdonnat xl: don't output (null) target in domxml-to-native When converting a domain xml containing a CDROM device without any attached source, don't add a target=(null) to the libxl config disk definition: xen doesn't like it at all and would fail to start the domain. 2016-10-18 Pavel Hrdina qemu: always generate the same alias for tls-creds-x509 object There was inconsistency between alias used to create tls-creds-x509 object and alias used to link that object to chardev while hotpluging. Hotplug ends with this error: error: Failed to detach device from channel-tcp.xml error: internal error: unable to execute QEMU command 'chardev-add': No TLS credentials with id 'objcharchannel3_tls0' In XML we have for example alias "serial0", but on qemu command line we generate "charserial0". The issue was that code, that creates QMP command to hotplug chardev devices uses only the second alias "charserial0" and that alias is also used to link the tls-creds-x509 object. This patch unifies the aliases for tls-creds-x509 to be always generated from "charserial0". 2016-10-18 Pavel Hrdina qemu_command: create prefixed alias to separate variable Instead of typing the prefix every time we want to append parameters to qemu command line use a variable that contains prefixed alias. qemu_alias: introduce qemuAliasChardevFromDevAlias helper qemu_hotplug: fix crash in hot(un)plugging chardev devices We need to make sure that the chardev is TCP. Without this check we may access different part of union and corrupt pointers. 2016-10-17 John Ferlan qemu: Remove unnecessary NULL arg check qemuDomainSecret{Disk|Hostdev}Prepare has a prototype that checks for ATTRIBUTE_NONNULL(1) for 'conn'. qemu: Add 'verify-peer=yes' test for chardev TCP TLS Missing the option to set verify-peer to yes qemu: Remove unnecessary cfg fetch/unref qemuProcessPrepareDomain has no need to fetch/unref the cfg, so remove it. 2016-10-17 Andrea Bolognani conf: Explain some code in more detail The code is entirely correct, but it still managed to trip me up when I first ran into it because I did not realize right away that VIR_PCI_CONNECT_TYPES_ENDPOINT was not a single flag, but rather a mask including both VIR_PCI_CONNECT_TYPE_PCI_DEVICE and VIR_PCI_CONNECT_TYPE_PCIE_DEVICE. In order to save the next distracted traveler in PCI Address Land some time, document this fact with a comment. Add a test case for the behavior as well. 2016-10-14 Laine Stump conf: restrict what type of buses will accept a pci-bridge A pci-bridge has *almost* the same rules as a legacy PCI endpoint device for where it can be automatically connected, and until now both had been considered identical. There is one pairing that is okay when specifically requested by the user (i.e. manual assignment), but we want to avoid it when auto-assigning addresses - plugging a pci-bridge directly into pcie-root (it is cleaner to plug in a dmi-to-pci-bridge, then plug the pci-bridge into that). In order to allow that difference, this patch makes a separate CONNECT_TYPE for pci-bridge, and uses it to restrict auto-assigned addresses for pci-bridges to be only on pci-root, pci-expander-bus, dmi-to-pci-bridge, or on another pci-bridge. NB: As with other discouraged-but-seem-to-work configurations (e.g. plugging a legacy PCI device into a pcie-root-port) if someone *really* wants to, they can still force a pci-bridge to be plugged into pcie-root (by manually specifying its PCI address.) 2016-10-14 John Ferlan util: Alter return value of virReadFCHost and fix mem leak https://bugzilla.redhat.com/show_bug.cgi?id=1357416 Rather than return a 0 or -1 and the *result string, return just the result string to the caller. Alter all the callers to handle the different return. As a side effect or result of this, it's much clearer that we cannot just assign the returned string into the scsi_host wwnn, wwpn, and fabric_wwn fields - rather we should fetch a temporary string, then as long as our fetch was good, VIR_FREE what may have been there, and STEAL what we just got. This fixes a memory leak in the virNodeDeviceCreateXML code path through find_new_device and nodeDeviceLookupSCSIHostByWWN which will continually call nodeDeviceSysfsGetSCSIHostCaps until the expected wwnn/wwpn is found in the device object capabilities. 2016-10-14 Martin Kletzander conf: Honour flags in virDomainPanicDefParseXML Without them we're keeping even for inactive XML. schema: Allow alias for panic device As with all other devices, it's not part of 'address'. 2016-10-14 Michal Privoznik qemu_hotplug: Support interface type of vhost-user hotplug https://bugzilla.redhat.com/show_bug.cgi?id=1366108 There are couple of things that needs to be done in order to allow vhost-user hotplug. Firstly, vhost-user requires a chardev which is connected to vhost-user bridge and through which qemu communicates with the bridge (no acutal guest traffic is sent through there, just some metadata). In order to generate proper chardev alias, we must assign device alias way sooner. Then, because we are plugging the chardev first, we need to do the proper undo if something fails - that is remove netdev too. We don't want anything to be left over in case attach fails at some point. 2016-10-14 Michal Privoznik qemuBuildHostNetStr: Support VIR_DOMAIN_NET_TYPE_VHOSTUSER https://bugzilla.redhat.com/show_bug.cgi?id=1366505 So far, this function lacked support for VIR_DOMAIN_NET_TYPE_VHOSTUSER leaving callers to hack around the problem by constructing the command line on their own. This is not ideal as it blocks hot plug support. 2016-10-14 Michal Privoznik qemuBuildVhostuserCommandLine: Unify -netdev creation Currently, what we do for vhost-user network is generate the following part of command line: -netdev type=vhost-user,id=hostnet0,chardev=charnet0 There's no need for 'type=' it is the default. Drop it. 2016-10-14 Michal Privoznik qemuBuildVhostuserCommandLine: Reuse qemuBuildChrChardevStr There's no need to reinvent the wheel here. We already have a function to format virDomainChrSourceDefPtr. It's called qemuBuildChrChardevStr(). Use that instead of some dummy virBufferAsprintf(). qemuBuildChrChardevStr: Introduce @nowait argument This alone makes not much sense. But the aim is to reuse this function in qemuBuildVhostuserCommandLine() where 'nowait' is not supported for vhost-user devices. qemuBuildHostNetStr: Explicitly enumerate net types We tend to prevent using 'default' in switches. And it is for a good reason - control may end up in paths we wouldn't want for new values. In this specific case, if qemuBuildHostNetStr is called over VIR_DOMAIN_NET_TYPE_VHOSTUSER it would produce meaningless output. Fortunately, there no such call yet. qemuDomainAttachNetDevice: Explicitly list allowed types for hotplug Instead of blindly claim support for hot-plugging of every interface type out there we should copy approach we have for device types: white listing supported types and explicitly error out on unsupported ones. For instance, trying to hotplug vhostuser interface results in nothing usable from guest currently. vhostuser typed interfaces require additional work on our side. qemuDomainAttachNetDevice: Move hostdev handling a bit further The idea is to have function that does some checking at its beginning and then have one big switch for all the interface types it supports. qemuBuildInterfaceCommandLine: Move from if-else forest to switch qemuBuildInterfaceCommandLine: Move vhostuser handling a bit further The idea is to have function that does some checking of the arguments at its beginning and then have one big switch for all the interface types it supports. Each one of them generating the corresponding part of the command line. qemuBuildInterfaceCommandLine: Move hostdev handling a bit further The idea is to have function that does some checking of the arguments at its beginning and then have one big switch for all the interface types it supports. Each one of them generating the corresponding part of the command line. virDomainNetGetActualType: Return type is virDomainNetType This function for some weird reason returns integer instead of virDomainNetType type. It is important to return the correct type so that we know what values we can expect. virDomainNetDefParseXML: Realign There are couple of formatting issues. No functional change though. virLogDefineOutputs: Fix build without syslog.h Not every system out there has syslog, that's why we check for it in our configure script. However, in 640b58abdf while fixing another issue, some variables and functions are called that are defined only when syslog.h is present. But these function calls/variables were not guarded by #ifdef-s. 2016-10-14 Peter Krempa schema: smbios: allow any strings The smbios docs allow any string to be passed and libvirt does not really do any validation on them. Allow passing any string. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1373535 2016-10-14 Peter Krempa qemu: command: escape smbios entry strings We pass free-form strings from the users to qemu, thus we need escape commas since they are passed to qemu monitor. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1373535 2016-10-14 Peter Krempa qemu: command: Don't bother reporting errors in smbios formatters qemuBuildSmbiosBiosStr and qemuBuildSmbiosSystemStr return NULL if there's nothing to format on the commandline. Reporting errors from buffer creation doesn't make sense since it would be ignored. qemu: command: Fix up coding style of smbios commandine formatters 2016-10-13 Michal Privoznik src: Treat PID as signed This initially started as a fix of some debug printing in virCgroupDetect. However it turned out that other places suffer from the similar problem. While dealing with pids, esp. in cases where we cannot use pid_t for ABI stability reasons, we often chose an unsigned integer type. This makes no sense as pid_t is signed. Also, new syntax-check rule is introduced so we won't repeat this mistake. 2016-10-13 Ivan Baldo virsh: Be explicit that vol-resize is an offline operation. It is stated in the manual already, so let's mention the same thing in the description to avoid causing problems. 2016-10-13 Michal Privoznik testNodeGetCellsFreeMemory: Fix off by one error Consider the following scenario: virsh # freecell --all 0: 2048 KiB 1: 4096 KiB -------------------- Total: 6144 KiB virsh # freecell 0 0: 2048 KiB virsh # freecell 1 1: 4096 KiB And now before this change: virsh # freecell 2 After this change: virsh # freecell 2 error: invalid argument: Range exceeds available cells 2016-10-13 Michal Privoznik testOpenDefault: Rename loop variable We have inclination to calling our loop variables i, j, k, not u. 2016-10-12 Nitesh Konkar virsh: Fix typos in manpage virsh: Fix commas in manpage to enhance readability. 2016-10-12 Pavel Hrdina qemu_command: add support to use virtio as secondary video device Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1369633 qemu_command: introduce enum of secondary models for video device There are two video devices with models without VGA compatibility mode. They are primary used as secondary video devices, but in some cases it is required to use them also as primary video devices. qemu_command: cleanup qemuBuildVideoCommandLine qemu_command: properly detect which model to use for video device This improves commit 706b5b6277 in a way that we check qemu capabilities instead of what architecture we are running on to detect whether we can use *virtio-vga* model or not. This is not a case only for arm/aarch64. 2016-10-12 Pavel Hrdina qemu_capabilities: check for existence of virtio-vga Commit 21373feb added support for primary virtio-vga device but it was checking for virtio-gpu. Let's check for existence of virtio-vga if we want to use it. Virtio video device is currently represented by three different models *virtio-gpu-device*, *virtio-gpu-pci* and *virtio-vga*. The first two models are tied together and if virtio video devices is compiled in they both exist. However, the *virtio-vga* model doesn't have to exist on some architectures even if the first two models exist. So we cannot group all three together. 2016-10-12 Pavel Hrdina qemu_command: pass only video device to qemuBuildVgaVideoCommand qemu_command: separate code for video device via -vga attribute 2016-10-12 Pavel Hrdina qemu_process: always check capabilities for video devices Before this patch we've checked qemu capabilities for video devices only while constructing qemu command line using "-device" option. Since we support qemu only if "-device" option is present we can use the same capabilities to check also video devices while using "-vga" option to construct qemu command line. 2016-10-12 Pavel Hrdina qemu_process: move video validation out of qemu_command Runtime validation that depend on qemu capabilities should be moved into qemuProcessStartValidateXML. 2016-10-12 Pavel Hrdina qemu_domain: move video validation out of qemu_command All definition validation that doesn't depend on qemu capabilities and was allowed previously as valid definition should be placed into qemuDomainDefValidate. The check whether video type is supported or not was based on an enum that translates type into model. Use switch to ensure that if new video type is added, it will be properly handled. 2016-10-12 Pavel Hrdina qemu_capabilities: detect properties for virtio-gpu-device qemu_capabilities: rename QEMU_CAPS_VIRTIO_GPU_VIRGL We generally uses QEMU_CAPS_DEVICE_$NAME to probe for existence of some device and QEMU_CAPS_$NAME_$PROP to probe for existence of some property of that device. 2016-10-12 Pavel Hrdina qemu_capabilities: mark QEMU_CAPS_VGA_QXL capability as deprecated If QEMU in question supports QMP, this capability is set if QEMU_CAPS_DEVICE_QXL was set based on existence of "-device qxl". If libvirt needs to parse *help*, because there is no QMP support, it checks for existence of "-vga qxl", but it also parses output of "-device ?" and sets QEMU_CAPS_DEVICE_QXL too. Now that libvirt supports only QEMU that has "-device" implemented it's safe to drop this capability and stop using it. 2016-10-12 Pavel Hrdina qemu_capabilities: join capabilities for qxl and qxl-vga devices This patch simplifies QEMU capabilities for QXL video device. QEMU exposes this device as *qxl-vga* and *qxl* and they are both the same device with the same set of parameters, the only difference is that *qxl-vga* includes VGA compatibility. Based on QEMU code they are tied together so it's safe to check only for presence of only one of them. This patch also removes an invalid test case "video-qxl-sec-nodevice" where there is only *qxl-vga* device and *qxl* device is not present. 2016-10-12 Pavel Hrdina tests: fix some QXL capability combinations that don't make sense If one of QEMU_CAPS_DEVICE_QXL_VGA or QEMU_CAPS_DEVICE_QXL is set the other one will always be set as well because both devices are tied together in QEMU. The change of args files is caused by the presence of capability QEMU_CAPS_DEVICE_VIDEO_PRIMARY which means it's safe to use "-device qxl-vga" instead of "-vga qxl", see commit (e3f2686b) and by the fact that if QEMU_CAPS_VGA_QXL is set QEMU_CAPS_DEVICE_QXL_VGA and QEMU_CAPS_DEVICE_QXL would be set too (since we support only qemu with "-device" option). 2016-10-12 Pavel Hrdina qemu_command: remove xenner leftover from video device code Qemu supports *xen* video device only with XEN and this code was part of xenner code. We dropped support for xenner in commit de9be0a. Before this patch if you used 'xen' video type you ended up with domain without any video device at all. Now we don't allow to start such domain. 2016-10-12 Pavel Hrdina qemu_process: move qemuProcessStartValidateGraphics to correct place 2016-10-12 Peter Krempa virsh: vcpuinfo: Report proper vcpu numbers and data for offline VMs If the VM is offline virsh attempted to at least report the pinning information for the VM. This would not work properly now that the vcpus can be sparse. Fix it by getting the vcpu states from the XML. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1375920 2016-10-12 Peter Krempa virsh: Extract fallback handling in cmdVcpuinfo Put it into a separate function so that more fallback handling can be added without making a mess. virsh: Extract cpumap formatting in cmdVcpuinfo cmdVcpuinfo will be split in upcomming patches thus extract the common code that formats pinning cpumaps for the vcpus. virsh: Fix xpath queries for retrieving vcpu count The fallback code used if virDomainGetVcpusFlags is not supported used wrong XPath queries and basically did not work at all. Fix them to point to the element instead of which was not present until lately. virsh: domain: Fix broken indentation in virshCPUCountCollect I managed to space most of the code by 5 spaces instead of 4 when orignally implementing this function. util: bitmap: Make bitmaps const in virBitmapNewData and virBitmapDataToString The functions just read the passed pointer so it can be marked as const. 2016-10-12 Andrea Bolognani libvirt-guests: Weaken dependency on libvirtd The Requires relationship is very strong, in that it prevents a unit from running unless all the units it Requires are running as well. This turns out to be a problem because we want to be able to restart libvirtd at any time without having libvirt-guests suspend or shutdown running guests. Turn the Requires relationship into a Wants relationship: this way starting libvirt-guests will cause systemd to (attempt to) start libvirtd as well, but stopping or restarting libvirtd will not alter libvirt-guests' running state. 2016-10-12 Martin Kletzander qemu: Disable migration with ivshmem It was never safe anyway and as such shouldn't have been enabled in the first place. Future patches will allow hot-(un)pluging of some ivshmem devices as a workaround. conf: Don't complicate find loop conf: Fix virDomainShmemDefFind Due to the switch of parameters in a call to virDomainShmemDefEquals() no device was found when looking for device with all the information except address. Also fix the indentation. Clean timer in virObjectEventStateFlush If the last event callback is unregistered while the event loop is dispatching, it is only marked as deleted, but not removed. The number of callbacks is more than zero in that case, so the timer is not removed. Because it can be removed in this function now (but also accessed afterwards so that we set 'isDispatching = false' and have it locked), we need to temporarily increase the reference counter of the state for the duration of this function. De-duplicate code into virObjectEventStateCleanupTimer() There is a repeating pattern of code that removes the timer if it's not needed. So let's move it to a new function. We'll also use it later. Reference state when using it as opaque There should be one more reference because it is being kept in the list of callbacks as an opaque. We also unref it properly using virObjectFreeCallback. Change virDomainEventState to virObjectLockable This way we get reference counting and we can get rid of locking function. 2016-10-12 Corey S. McQuay qemu: migration: Disallow migration of read only disk Currently Libvirt allows attempts to migrate read only disks. Qemu cannot handle this as read only disks cannot be written to on the destination system. The end result is a cryptic error message and a failed migration. This patch causes migration to fail earlier and provides a meaningful error message stating that migrating read only disks is not supported. Reviewed-by: Jason J. Herne Reviewed-by: Boris Fiuczynski 2016-10-11 Peter Krempa qemu: Reuse virDomainDeGetVcpusTopology to calculate total vcpu count Rather than multiplying sockets, cores, and threads use the new helper for getting the vcpu count resulting from the topology. 2016-10-11 Peter Krempa conf: Sanitize cpu topology numbers Make sure that the topology results into a sane number of cpus (up to UINT_MAX) so that it can be sanely compared to the vcpu count of the VM. Additionally the helper added in this patch allows to fetch the total number the topology results to so that it does not have to be reimplemented later. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1378290 2016-10-10 John Ferlan util: Remove need for local 'nelems' Since it's only used in loop - just go direct. tests: Need to initialize data If not initialized and the virAsprintf to jsonreply or fulllablel fails, then the call to qemuMonitorTestFree will take stack data. 2016-10-10 John Ferlan vsh: Fix some issues in auto completion code 1. Move the declaration of const vshCmdDef *help - it should be at the top of the "if" rather than in the middle. 2. Change a comparison from && to || - without doing so we could crash on commands like 'virsh list' which would allow completion of some non -- option based on whatever was found in the current working directory and then as soon as that was completed, the next would crash since "opt" would be returned as NULL, but the check was dereferencing "&& opt->type" 3. Before dereferencing opt->completer, be sure opt isn't NULL. 2016-10-10 John Ferlan util: Resolve memory leaks in virLogParse{Output|Filter} In both virLogParseOutput and virLogParseFilter, rather than returning NULL, goto cleanup since it's possible that for each the first condition passes, but the || condition doesn't and thus we leak memory. tests: Prefer virGetLastErrorMessage in testSELinuxLabeling Yet another case of not needing virGetLastError processing conf: Remove incorrect check when encoding shmem audit message Remove the !size check since size is initialized to NULL and thus causing the condition to always be true 2016-10-10 Martin Wilck network: add dnsmasq option 'dhcp-authoritative' The dnsmasq man page recommends that dhcp-authoritative "should be set when dnsmasq is definitely the only DHCP server on a network". This is the case for libvirt-managed virtual networks. The effect of this is that VMs that fail to renew their DHCP lease in time (e.g. if the VM or host is suspended) will be able to re-acquire the lease even if it's expired, unless the IP address has been taken by some other host. This avoids various annoyances caused by changing VM IP addresses. 2016-10-10 Martin Kletzander Don't update timer if there's none. Sometimes virObjectEventStateFlush can be called without timer (if the last event was unregistered right when the timer fired). There is a check for timer == -1, but that triggers warning and other log messages, which is unnecessary. 2016-10-10 Stefan Bader tools: Exclude Xen dom0 from libvirt-guests.sh list With newer versions of libvirt Domain-0 is again visible in the list of running guests but it should not be considered as a guest for shutdown or suspend. Signed-off-by Stefan Bader 2016-10-10 Stefan Bader tools: Ignore newlines in libvirt-guests.sh guest list The list file expects all guest UUIDs on the same line as the URI which the guests run on. This does not happen when the list is echo'ed in quotes. When stripping the quotes, newlines get transformed into spaces. Without this, only the first guest on the list is actually handled. Based on a fix by Omar Siam Bug-Ubuntu: http://bugs.launchpad.net/bugs/1591695 2016-10-10 Luyao Huang examples: Distribute all systemtap scripts. Due to a missing entries in Makefile, we were not distributing all the systemtap scripts we have. 2016-10-10 Erik Skultety virlog: Split parsing and setting priority Handling of outputs and filters has been changed in a way that splits parsing and defining. Do the same thing for logging priority as well, this however, doesn't need much of a preparation. virlog: Remove functions that aren't used anywhere anymore This is mainly virLogAddOutputTo* which were replaced by virLogNewOutputTo* and the previously poorly named ones virLogParseAndDefine* functions. All of these are unnecessary now, since all the original callers were transparently switched to the new model of separate parsing and defining logic. daemon: Split filter parsing and filter defining Similar to outputs, parser should do parsing only, thus the 'define' logic is going to be stripped from virLogParseAndDefineFilters by replacing calls to this method to virLogSetFilters instead. daemon: Split output parsing and output defining Since virLogParseAndDefineOutputs is going to be stripped from 'output defining' logic, replace all relevant occurrences with virLogSetOutputs call to make the change transparent to all original callers (daemons mostly). virlog: Introduce virLogSetFilters This method will eventually replace virLogParseAndDefineFilters which currently does both parsing and defining. virlog: Introduce virLogSetOutputs This API is the entry point to output modification of the logger. Currently, everything is done by virLogParseAndDefineOutputs. Parsing and defining will be split into two operations both handled by this method transparently. virlog: Introduce virLogParseFilters Abstraction added over parsing a single filter. The method parses potentially a set of logging filters, while adding each filter logging object to a caller-provided array. virlog: Introduce virLogParseOutputs Another abstraction added on the top of parsing a single logging output. This method takes and parses the whole set of outputs, adding each single output that has already been parsed into a caller-provided array. If the user-supplied string contained duplicate outputs, only the last occurrence is taken into account (all the others are removed from the list), so we silently avoid duplicate logs. virlog: Introduce virLogParseFilter Same as for outputs, introduce a new method, that is basically the same as virLogParseAndDefineFilter with the difference that it does not define the filter. It rather returns a newly created object that needs to be inserted into a list and then defined separately. virlog: Introduce virLogParseOutput Introduce a method to parse an individual logging output. The difference compared to the virLogParseAndDefineOutput is that this method does not define the output, instead it makes use of the virLogNewOutputTo* methods introduced in the previous patch and just returns the virLogOutput object that has to be added to a list of object which then can be defined as a whole via virLogDefineOutputs. The idea remains still the same - split parsing and defining of the logging primitives (outputs, filters). Additionally, since virLogNewOutputTo* methods are now finally used, ATTRIBUTE_UNUSED can be successfully removed from the methods' definitions, since that was just to avoid compiler complaints about unused static functions. virlog: Take a special care of syslog when setting new set of log outputs Now that we're in the critical section, syslog connection can be re-opened by issuing openlog, which is something that cannot be done beforehand, since syslog keeps its file descriptor private and changing the tag earlier might introduce a log inconsistency if something went wrong with preparing a new set of logging outputs in order to replace the existing one. virlog: Introduce virLogNewOutputTo* as a replacement for virLogAddOutputTo* Continuing with the effort to split output parsing and defining, these new functions return a logging object reference instead of defining the output. Eventually, these functions will replace the existing ones (virLogAddOutputTo*) which will then be dropped. virlog: Introduce virLogDefineFilters Prepare a method that only defines a set of filters. It takes a list of filters, preferably created by virLogParseFilters. The original set of filters is reset and replaced by the new user-provided set of filters. virlog: Introduce virLogDefineOutputs Prepare a method that only defines a set of outputs. It takes a list of outputs, preferably created by virLogParseOutputs. The original set of outputs is reset and replaced by the new user-provided set of outputs. virlog: Introduce virLogFindOutput Outputs are a bit trickier than filters, since the user(config)-specified set of outputs can contain duplicates. That would lead to logging the same message twice. For compatibility reasons, we cannot just error out and forbid the daemon to start if we find duplicate outputs which do not make sense. Instead, we could silently take into account only the last occurrence of the duplicate output and remove all the previous ones, so that the logger will not try to use them when it is looping over all of its registered outputs. virlog: Introduce virLogFilterNew This method allocates a new filter object which it then returns back to caller. virlog: Introduce virLogOutputNew In order to later split output parsing and output defining, introduce a new function which will create a new virLogOutput object which the parser will insert into a list with the list being eventually defined. virlog: Store the journald fd within the output object There is really no reason why we could not keep journald's fd within the journald output object the same way as we do for regular file-based outputs. By doing this we later won't have to special case the journald-based output (due to the fd being globally shared) when replacing the existing set of outputs with a new one. Additionally, by making this change, we don't need the virLogCloseJournald routine anymore, plain virLogCloseFd will suffice. virlog: Rename virLogParse* to virLogParseAndDefine* Right now virLogParse* functions are doing both parsing and defining of filters and outputs which should be two separate operations. Since the naming is apparently a bit poor this patch renames these functions to virLogParseAndDefine* which eventually will be replaced by virLogSet*. Additionally, virLogParse{Filter,Output} will be later (after the split) reused, so that these functions do exactly what the their name suggests. virlog: Remove unused macro IS_SPACE During first stage of virlog.c refactor, commit 0b231195 forgot to remove the macro definition along with its usage. 2016-10-10 Chen Hanxiao virsh domdisplay: introduce '--all' for showing all possible graphical displays For one VM, it could had more than one graphical display. Such as we coud add both vnc and spice display to a VM. This patch introduces '--all' for showing all possible graphical display of a active VM. Reviewed-by: Michal Privoznik 2016-10-07 John Ferlan util: Check/ignore already disabled event If the event is already disabled, then don't bother with setting it disabled again. Causes unnecessary error on systems that don't support the feature anyway. util: Clear up some perf error messages Make it clearer that the perf event is based/for the host cpu and use the virPerfEventTypeToString to convert the type to a string 2016-10-07 John Ferlan docs: Alter descriptions of perf cpu_cycles https://bugzilla.redhat.com/show_bug.cgi?id=1381714 Alter the descriptions to match what the cpu_cycles actually is 2016-10-07 Daniel P. Berrange qemu: fix command line building for iommu devices The intel-iommu device has existed since QEMU 2.2.0, but it was only possible to create it with -device since QEMU 2.7.0, thanks to: commit 621d983a1f9051f4cfc3f402569b46b77d8449fc Author: Marcel Apfelbaum Date: Mon Jun 27 18:38:34 2016 +0300 hw/iommu: enable iommu with -device Use the standard '-device intel-iommu' to create the IOMMU device. The legacy '-machine,iommu=on' can still be used. The libvirt capability check & command line formatting code is thus broken for all QEMU versions 2.2.0 -> 2.6.0 inclusive. This fixes it to use iommu=on instead. 2016-10-07 Marko Myllynen docs: Add PCP as monitoring application This has been announced on the list: https://www.redhat.com/archives/libvir-list/2016-October/msg00187.html 2016-10-06 Erik Skultety private_syms: add virLogFilterListFree to libvirt_private.syms Commit 660468b1 forgot to add it, so let's add it now to prevent future linker issues. 2016-10-06 Andrea Bolognani m4: Drop PKG_PROG_PKG_CONFIG compatibility code This was needed for RHEL 4 vintage distributions, which we haven't supported for a long time now. 2016-10-06 Martin Kletzander virt-yajl: Fix detection of yajl requirements Running the output of qemu -help doesn't make any sense. We should be looking for libvirt being mentioned in the output. This worked by accident, let's make it work as expected it to. 2016-10-06 Ján Tomko docs/Makefile.am: build hvsupport.html earlier This file requires three steps instead of two. Move it earlier in the list of targets to avoid waiting for it. 2016-10-06 Ján Tomko docs/Makefile.am: remove redundant variables Remove DOC_SOURCE_DIR, introduced by and unused since commit b325d74. PERL is already detected in configure.ac. 2016-10-06 Peter Krempa qemu: monitor: Properly configure backend for UDP chardevs Since introduction of chardev hotplug the code was wrong for the UDP case and basically created a TCP socket instead. Use proper objects and type for UDP. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1377602 2016-10-06 Peter Krempa qemu: monitor: Simplify construction of chardev backends 2016-10-06 Peter Krempa tests: qemumonitorjsontest: Do some actual testing in qemuMonitorJSONTestAttachChardev Until now the test was rather useless since it didn't check the arguments formatted and didn't use properly configured chardev objects. Add the expected arguments and instrument the test to validate them. Modify some test cases to actually add valid data. Note that the UDP test data is currently wrong due to a bug. 2016-10-06 Peter Krempa tests: qemumonitorjson: Don't do multiple tests in one virTestRun The chardev attach test would do all the tests in one virTestRun instance. If one sub-test failed then the test would report failure improperly and the error would be hard to debug since the error pointer was overwritten. tests: qemu: Add support for testing aguments on monitor verbatim Add code that takes a string and matches it against the data passed as arguments from qemu. This is a simpler version of qemuMonitorTestAddItemParams. conf: Sanitize formatting of UDP chardev source Use much simpler logic to determine parts of the code to print. 2016-10-06 John Ferlan conf: Add a formatting macro for all the blkiotune values Rather than copy-paste - use a macro Unfortunately due to how the RNG schema was written keeping the 'value' and 'value'_max next to each other in the XML causes a schema failure, so the FORMAT has to write out singly rather than optimizing to write out both values at once 2016-10-06 John Ferlan qemu: Adjust how supportMaxOptions is used. We're about to add more options, let's avoid having multiple if-then-else which each try to set up the qemuMonitorJSONMakeCommand call with all the parameters it knows about. Instead, use the fact that when a NULL is found in the argument list that processing of the remaining arguments stops and just have call. 2016-10-06 John Ferlan qemu: Convert from shorthand to longer throttling names We're about to add 6 new options and it appears (from testing) one cannot utilize both the shorthand (alias) and (much) longer names for the arguments. So modify the command builder to use the longer name and of course alter the test output .args to have the similarly innocuous long name. Also utilize a macro to build that name makes it so much more visually appealing and saves a few characters or potential cut-n-paste issues. 2016-10-06 John Ferlan tests: Add blkdeviotune-max xml2xmltest It was missing... Also since I'm using the soft link from qemuxml2xmloutdata to the qemuxml2argvdata file, modify the output file to have the necessary
elements plus the mouse and keyboard. include: Update description for max params The upstream qemu commit 'dce13204' changed the wording just slightly to add 'in bursts' essentially. Just following that model here. docs: Fix typo in libvirt-domain.h parameter description Change Marco to Macro 2016-10-05 Laine Stump qemu: allow 32 slots on pcie-expander-bus, not just 1 When I added support for the pcie-expander-bus controller in commit bc07251f, I incorrectly thought that it only had a single slot available. Actually it has 32 slots, just like the root complex aka pcie-root (the part that I *did* get correct is that unlike pcie-root a pcie-expander-bus doesn't allow any integrated endpoint devices - only pcie-root-ports and dmi-to-pci-controllers are allowed). 2016-10-05 Jiri Denemark vsh: Fix warnings in command line completer GCC complained that vsh.c: In function 'vshReadlineOptionsGenerator': vsh.c:2622:29: warning: unused variable 'opt' [-Wunused-variable] const vshCmdOptDef *opt = &cmd->opts[list_index]; ^ vsh.c: In function 'vshReadlineParse': vsh.c:2830:44: warning: 'opt' may be used uninitialized in this function [-Wmaybe-uninitialized] completed_list = opt->completer(autoCompleteOpaque, 2016-10-05 John Ferlan qemu: Create helper qemuMonitorJSONGetBlockDevDevice This will fetch "this device" from the recently returned 'dev' and perform common error checking for the paths that call it. qemu: Create helper qemuMonitorJSONGetBlockDev This will grab the 'dev' from devices and do the common validation checks. 2016-10-05 John Ferlan qemu: Create common code for JSON "query-block" call Reduce some cut-n-paste code by creating common helper. Make use of the recently added virJSONValueObjectStealArray to grab the devices list as part of the common code (we we can Free the reply) and return devices for each of the callers to continue to parse. NB: This also adds error checking to qemuMonitorJSONDiskNameLookup 2016-10-05 John Ferlan util: Introduce virJSONValueObjectStealArray Provide the Steal API for any code paths that will desire to grab the object array and then free it afterwards rather than relying to freeing the whole chain from the reply. rbd: Move the encryption check in build No sense opening a connection only to fail because we don't support the type of build being attempted. rbd: Change to using heap allocated state contexts Rather than use stack allocated state context pointers, let's allocate and free the state context pointer. In doing so, we'll shrink the code a bit since many routines perform the same initialization sequence. 2016-10-05 John Ferlan rbd: Change virStorageBackendRBDCloseRADOSConn to be static void Since none of the callers check the status, let's just alter it to a static void. While we're at it - scrap the local runtime variable and just do the math in the VIR_DEBUG directly. 2016-10-05 Peter Krempa qemu: Allow making vcpus hotpluggable with virDomainSetVcpusFlags Implement support for VIR_DOMAIN_VCPU_HOTPLUGGABLE so that users can choose to make vcpus added by the API removable. lib: Introduce VIR_DOMAIN_VCPU_HOTPLUGGABLE for virDomainSetVcpusFlags For compatibility reasons virDomainSetVcpus needs to add vcpus as non hotpluggable which means that the users will not be able to unplug it after the VM has started. Add a flag that will allow to tell the API that the unpluggable vcpus are okay. 2016-10-05 Peter Krempa qemu: attach: Close monitor socket on connection failure If attaching to a qemu process fails after opening the monitor socket libvirt does not clean up the monitor. As the monitor also holds a reference to the domain object the qemu attach API basically leaks it. QEMU also does not interact on a second monitor connection and thus a further attempt to attach to it would lock up. Prevent libvirt from leaking the monitor by explicitly closing it. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1378401 2016-10-05 Peter Krempa qemu: Don't strictly require JSON monitor for vCPU detection Attaching to a existing qemu process allows to get us into a situation when qemu is new enough to have JSON monitor and new vCPU hotplug but the json monitor is not used. The vCPU detection code would require it though. This broke attaching to qemu processes. Make the condition less strict and just skip the vCPU hotplug detection if JSON monitor is not available. Resolves one of the symptoms in: https://bugzilla.redhat.com/show_bug.cgi?id=1378401 2016-10-05 Nehal J Wani Don't drop expired lease while reading custom leases file Libvirt, on its own, shouldn't decide whether an expired lease should stay in the custom leases database or not. It should rather rely on the 'DEL' event from dnsmasq. Reviewed-by: Michal Privoznik 2016-10-05 Nehal J Wani NSS: Add explicit check to not report expired lease The NSS module shouldn't rely on custom leases database to not have entries for leases which have expired. Reviewed-by: Michal Privoznik 2016-10-04 John Ferlan remote: Increase bound limit for virDomainGetBlockIoTune We are about to add 6 new values to fetch. This will put us over the current limit of 16 (we're at 13 now). Once there are more than 16 parameters, this will affect existing clients that attempt to fetch blockiotune config values for the domain from the remote host since the server side has no mechanism to determine whether the capability for the emulator exists and thus would attempt to return all known values from the persistentDef. If attempting to fetch the blockiotune values from a running domain, the code will check the emulator capabilities and set maxparams (in qemuDomainGetBlockIoTune) appropriately. On the client side of the remote connection, it uses this constant in xdr_remote_domain_get_block_io_tune_ret and virTypedParamsDeserialize calls, so if a remote server returns more than 16 parameters, then the client will fail with "Unable to decode message payload". 2016-10-04 John Ferlan remote: Fix erroneous usage of constant The REMOTE_DOMAIN_MEMORY_PARAMETERS_MAX was erroneously used in the remoteDomainBlockStatsFlags and remoteDomainGetBlockIoTune calls. Change the constant to be the right one. Fortunately, all 3 are defined as 16. 2016-10-04 John Ferlan Post-release version bump to 2.4.0 2016-10-04 Daniel Veillard Release of libvirt-2.3.0 * docs/news.html.in: updated for release * po/*.po*: regenerated 2016-10-04 Michal Privoznik Revert "conf: Skip post parse callbacks when creating copy" This breaks vCPU hotplug, because when starting a domain, we create a copy of domain definition (which becomes live XML) and during the post parse callbacks we might adjust some tunings so that vCPU hotplug is possible. This reverts commit 581b7756af18dcf84b57d9947978725d2dfbfc18. 2016-10-04 Michal Privoznik Revert "domain_conf: Introduce VIR_DOMAIN_DEF_PARSE_SKIP_POST_PARSE" This breaks vCPU hotplug, because when starting a domain, we create a copy of domain definition (which becomes live XML) and during the post parse callbacks we might adjust some tunings so that vCPU hotplug is possible. This reverts commit c0f90799bc7fa4b690ead6a592806378a243873c. 2016-09-30 Peter Krempa qemu: vcpu: Clear vcpu order information rather than making it invalid Certain operations may make the vcpu order information invalid. Since the order is primarily used to ensure migration compatibility and has basically no other user benefits, clear the order prior to certain operations and document that it may be cleared. All the operations that would clear the order can still be properly executed by defining a new domain configuration rather than using the helper APIs. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1370357 2016-09-30 Peter Krempa qemu: Fix coldplug of vcpus virDomainDefSetVcpus was not designed to handle coldplug of vcpus now that we can set state of vcpus individually. Introduce qemuDomainSetVcpusConfig that properly handles state changes of vcpus when coldplugging so that invalid configurations are not created. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1375939 2016-09-30 Peter Krempa qemu: process: Enforce 'vcpu' order range to <1,maxvcpus> The current code that validates duplicate vcpu order would not work properly if the order would exceed def->maxvcpus. Limit the order to the interval described. qemu: process: Don't use shifted indexes for vcpu order verification Allocate a one larger bitmap rather than shifting the indexes back to zero. 2016-09-30 Peter Krempa qemu: process: Fix off-by-one in vcpu order duplicate error message The bitmap indexes for the order duplicate check are shifted to 0 since vcpu order 0 is not allowed. The error message doesn't need such treating though. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1370360 2016-09-29 Laine Stump docs: correct version requirements for When support was added for the kvm hidden='on' attribute in commit d07116, the version requirement was listed as "2.1.0 (QEMU only)". However, this was added when libvirt was at version 1.2.8 - it is *QEMU* that must be at version 2.1.0 or later. This went unnoticed for a very long time (over 2 years). Then a week or two ago a new Windows convert in the #virt channel on OFTC was told he needed to use this feature (to prevent nvidia drivers in a guest from refusing to work due to being run in a virtual machine). There was some problem with it being recognized and "someone" (it may have been me, or may have been someone else, I don't remember) pointed out that the documentation at http://www.libvirt.org/formatdomain.html says that it requires libvirt 2.1.0. The next several days were filled with agony as a new convert to Linux first tried to upgrade a Linux Mint host running their "LTS" version to something newer, then tried to install a libvirt build built for Ubuntu onto this, and later back to the old LTS Linux Mint. After this he tried building his own libvirt from source (with all the expected problems), and finally switched to Fedora. In the end it was hours and hours of everybody's lives that they will never get back. To now learn that he didn't need to do this (his original libvirt version was 1.3.3, so whatever his problem was, it was elsewhere) makes the pain all that much worse. To prevent this from happening again, this simple patch changes the version requirement for the kvm hidden attribute from "2.1.0 (QEMU only)" to "1.2.8 (QEMU 2.1.0)". 2016-09-29 Michal Privoznik sanlock: Properly init io_timeout https://bugzilla.redhat.com/show_bug.cgi?id=1292984 Hold on to your hats, because this is gonna be wild. In bd3e16a3 I've tried to expose sanlock io_timeout. What I had not realized (because there is like no documentation for sanlock at all) was very unusual way their APIs work. Basically, what we do currently is: sanlock_add_lockspace_timeout(&ls, io_timeout); which adds a lockspace to sanlock daemon. One would expect that io_timeout sets the io_timeout for it. Nah! That's where you are completely off the tracks. It sets timeout for next lockspace you will probably add later. Therefore: sanlock_add_lockspace_timeout(&ls, io_timeout = 10); /* adds new lockspace with default io_timeout */ sanlock_add_lockspace_timeout(&ls, io_timeout = 20); /* adds new lockspace with io_timeout = 10 */ sanlock_add_lockspace_timeout(&ls, io_timeout = 40); /* adds new lockspace with io_timeout = 20 */ And so on. You get the picture. Fortunately, we don't allow setting io_timeout per domain or per domain disk. So we just need to set the default used in the very first step and hope for the best (as all the io_timeout-s used later will have the same value). 2016-09-29 Michal Privoznik m4: Check for sanlock_write_lockspace Currently, we are checking for sanlock_add_lockspace_timeout which is good for now. But in a subsequent patch we are going to use sanlock_write_lockspace (which sets an initial value for io timeout for sanlock). Now, there is no reason to check for both functions in sanlock library as the sanlock_write_lockspace was introduced in 2.7 release and the one we are currently checking for in the 2.5 release. Therefore it is safe to assume presence of sanlock_add_lockspace_timeout when sanlock_write_lockspace is detected. Moreover, the macro for conditional compilation is renamed to HAVE_SANLOCK_IO_TIMEOUT (as it now encapsulates two functions). 2016-09-29 Michal Privoznik lock_driver_sanlock: Avoid global driver variable whenever possible Global variables are bad, we should avoid using them. 2016-09-29 Martin Kletzander qemu: Only use memory-backend-file with NUMA if needed If this reminds you of a commit message from around a year ago, it's 41c2aa729f0af084ede95ee9a06219a2dd5fb5df and yes, we're dealing with "the same thing" again. Or f309db1f4d51009bad0d32e12efc75530b66836b and it's similar. There is a logic in place that if there is no real need for memory-backend-file, qemuBuildMemoryBackendStr() returns 0. However that wasn't the case with hugepage backing. The reason for that was that we abused the 'pagesize' variable for storing that information, but we should rather have a separate one that specifies whether we really need the new object for hugepage backing. And that variable should be set only if this particular NUMA cell needs special treatment WRT hugepages. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1372153 2016-09-29 John Ferlan vsh: Write out history on "quit" or "exit" in interactive mode https://bugzilla.redhat.com/show_bug.cgi?id=1379895 Introduced by commit id '834c5720'. During the code motion and creation of vsh.c, the function 'vshDeinit()' in the (new) vsh.c was altered from whence it came in virsh.c such that calling 'vshReadlineDeinit(ctl)' was conditional on "ctl->imode". This causes a problem for the interactive running if the "quit" and "exit" commands are used because 'cmdQuit' will clear ctl->imode, thus when the interactive loop in main() of virsh.c exits because ctl->mode is clear and virshDeinit is called which calls vshDeinit, the history file is now not written. Conversely, if one had exited the interactive loop via pressing D the file would be created because loop control is broken on EOF and ctl->imode is not set to false. This patch will remove the conditional call to vshReadlineDeinit and restore the former behaviour. 2016-09-29 Roman Bogorodskiy bhyve: chase cpuCompareXML rename In commit 7f127de cpuCompareXML was renamed to virCPUCompareXML, so change the bhyve driver to use the new function and thus fix the build. 2016-09-28 Jim Fehlig doc: fix note about Xen credit scheduler Commit 6c504d6a added a note to the virsh man page about the deprecation of 'cap' and 'weight' settings for the credit scheduler. To this day, the default scheduler in Xen is credit and it supports setting 'cap' and 'weight'. Remove the deprecation notice from the note on the Xen credit scheduler. Reported-by: Volo M. 2016-09-28 Jim Fehlig libxl: fix param assignment in domainGetSchedulerParameters Due to a copy and paste error, the scheduler 'cap' parameter was over-writing the 'weight' parameter when preparing the return parameters in libxlDomainGetSchedulerParametersFlags. As a result, the scheduler weight was never shown when getting schedinfo and setting the weight failed as well virsh schedinfo testvm Scheduler : credit cap : 0 virsh schedinfo testvm --cap 50 --weight 500 Scheduler : credit error: invalid scheduler option: weight The obvious fix is to assign the 'caps' parameter to the correct item in the parameter list. Reported-by: Volo M. 2016-09-27 Joao Martins xlconfigtest: add test for channel conversion Acked-by: Jim Fehlig 2016-09-27 Joao Martins xenconfig: channels conversion support Add support for formating/parsing libxl channels. Syntax on xen libxl goes as following: channel=["connection=pty|socket,path=/path/to/socket,name=XXX",...] Acked-by: Jim Fehlig 2016-09-27 Joao Martins libxl: channels support And allow libxl to handle channel element which creates a Xen console visible to the guest as a low-bandwitdh communication channel. If type is PTY we also fetch the tty after boot using libxl_channel_getinfo to fetch the tty path. On socket case, we autogenerate a path if not specified in the XML. Path autogenerated is slightly different from qemu driver: qemu stores also on "channels/target" but it creates then a directory per domain with each channel target name. libxl doesn't appear to have a clear definition of private files associated with each domain, so for simplicity we do it slightly different. On qemu each autogenerated channel goes like: channels/target// Whereas for libxl: channels/target/- Should note that if path is not specified it won't persist, existing only on live XML, unless user had initially specified it. Since support for libxl channels only came on Xen >= 4.5 we therefore need to conditionally compile it with LIBXL_HAVE_DEVICE_CHANNEL. After this patch and having a qemu guest agent: $ cat domain.xml | grep -a1 channel | head -n 5 | tail -n 4 $ virsh create domain.xml $ echo '{"execute":"guest-network-get-interfaces"}' | socat stdio,ignoreeof unix-connect:/tmp/channel {"execute":"guest-network-get-interfaces"} {"return": [{"name": "lo", "ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "127.0.0.1", "prefix": 8}, {"ip-address-type": "ipv6", "ip-address": "::1", "prefix": 128}], "hardware-address": "00:00:00:00:00:00"}, {"name": "eth0", "ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "10.100.0.6", "prefix": 24}, {"ip-address-type": "ipv6", "ip-address": "fe80::216:3eff:fe40:88eb", "prefix": 64}], "hardware-address": "00:16:3e:40:88:eb"}, {"name": "sit0"}]} 2016-09-27 Joao Martins conf: add xen type for channels So far only guestfwd and virtio were supported. Add an additional for Xen as libxl channels create a Xen console visible to the guest. 2016-09-27 Jiri Denemark qemu: Fix crash in qemucapsprobe The qemucapsprobe helper calls virQEMUCapsNewForBinaryInternal with caps == NULL, causing the following crash: Program received signal SIGSEGV, Segmentation fault. #0 0x00007ffff788775f in virQEMUCapsInitHostCPUModel (qemuCaps=qemuCaps@entry=0x649680, host=host@entry=0x10) at src/qemu/qemu_capabilities.c:2969 #1 0x00007ffff7889dbf in virQEMUCapsNewForBinaryInternal (caps=caps@entry=0x0, binary=, libDir=libDir@entry=0x4033f6 "/tmp", cacheDir=cacheDir@entry=0x0, runUid=runUid@entry=4294967295, runGid=runGid@entry=4294967295, qmpOnly=true) at src/qemu/qemu_capabilities.c:4039 #2 0x0000000000401702 in main (argc=2, argv=0x7fffffffd968) at tests/qemucapsprobe.c:73 Caused by v2.2.0-182-g68c7011. 2016-09-27 Michal Privoznik qemuDomainUpdateDeviceConfig: Allow full disk update https://bugzilla.redhat.com/show_bug.cgi?id=1368417 So far, when it comes to 'virsh update-device --config' of disks we are limiting ourselves for just the disk source update and just for CDROMs and floppies. This makes no sense. Especially if you look around and see that we already allow full update to graphics and net devices. So let's just take whatever XML user wants to have there and replace our internal definition with it. 2016-09-27 Jim Fehlig libxl: find virDomainObj in libxlDomainShutdownThread libxl events are delivered to libvirt via the libxlDomainEventHandler callback registered with libxl. Documenation in $xensrc/tools/libxl/libxl_event.h states that the callback "may occur on any thread in which the application calls libxl". This can result in deadlock since many of the libvirt callees of libxl hold a lock on the virDomainObj they are working on. When the callback is invoked, it attempts to find a virDomainObj corresponding to the domain ID provided by libxl. Searching the domain obj list results in locking each obj before checking if it is active, and its ID equals the requested ID. Deadlock is possible when attempting to lock an obj that is already locked further up the call stack. Indeed, Max Ustermann reported an instance of this deadlock https://www.redhat.com/archives/libvir-list/2015-November/msg00130.html Guido Rossmueller also recently stumbled across it https://www.redhat.com/archives/libvir-list/2016-September/msg00287.html Fix the deadlock by moving the lookup of virDomainObj to the libxlDomainShutdownThread. After this patch, libxl events are enqueued on the libvirt side and processed by dedicated thread, avoiding the described deadlock. Reported-by: Max Ustermann Reported-by: Guido Rossmueller 2016-09-27 Jiri Denemark mingw: Package cputypes.rng for mingw32 too mingw: Package cputypes.rng spec: Package cputypes.rng 2016-09-27 Daniel P. Berrange qemu: make qemuGetCompressionProgram return int not an enum enum types are unsigned and the qemuGetCompressionProgram function can return -1 on error. It is therefore inappropriate to return an enum type. This fixes a build error where the internal 'ret' variable was used in a comparison with -1 ../../src/qemu/qemu_driver.c: In function 'qemuGetCompressionProgram': ../../src/qemu/qemu_driver.c:3280:5: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits] ../../src/qemu/qemu_driver.c:3289:5: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits] cc1: all warnings being treated as errors 2016-09-27 Nitesh Konkar Fix coding style issues. 2016-09-26 Michal Privoznik conf: Skip post parse callbacks when creating copy When creating a copy of virDomainDef we save ourselves the trouble of writing deep-copy functions and just format and parse back domain/device XML. However, the XML we are parsing was already fully formatted - there is no reason to run post parse callbacks (which fill in blanks - there are none!). domain_conf: Introduce VIR_DOMAIN_DEF_PARSE_SKIP_POST_PARSE This is an internal flag that prevents our two entry points to XML parsing (virDomainDefParse and virDomainDeviceDefParse) from running post parse callbacks. This is expected to be used in cases when we already have full domain/device XML and we are just parsing it back (i.e. virDomainDefCopy or virDomainDeviceDefCopy) qemuDomainDefAssignAddresses: Fetch caps from domain object Just like we did two commits ago, don't try to fetch capabilities for non-existing binary. Re-use the ones we have for running domain. qemuDomainDeviceDefPostParse: Fetch caps from domain object Just like we did two commits ago, don't try to fetch capabilities for non-existing binary. Re-use the ones we have for running domain. qemuDomainDefPostParse: Fetch qemuCaps from domain object We can't rely on def->emulator path. It may be provided by user as we give them opportunity to provide their own XML for migration. Therefore the path may point to just whatever binary (or even to a non-existent file). Moreover, this path is meant for destination, but the capabilities lookup is done on source. What we can do is to assume same capabilities for post parse callbacks as the running domain has. They will be used just to add some default models/controllers/devices/... anyway. conf: Extend virDomainDefAssignAddressesCallback for parseOpaque conf: Extend virDomainDeviceDefPostParse for parseOpaque Just like virDomainDefPostParseCallback has gained new parseOpaque argument, we need to follow the logic with virDomainDeviceDefPostParse. virDomainDefCopy: Introduce @parseOpaque argument We want to pass the proper opaque pointer instead of NULL to virDomainDefParseString. virDomainDefParse{File,String}: Introduce @parseOpaque argument We want to pass the proper opaque pointer instead of NULL to virDomainDefParse and subsequently virDomainDefParseNode too. virDomainDefParseNode: Introduce @parseOpaque argument We want to pass the proper opaque pointer instead of NULL to virDomainDefParseXML and subsequently virDomainDefPostParse too. virDomainDefPostParse: Introduce @parseOpaque argument Some callers might want to pass yet another pointer to opaque data to post parse callbacks. The driver generic one is not enough because two threads executing post parse callback might want to see different data (e.g. domain object pointer that domain def belongs to). 2016-09-26 Chen Hanxiao storage_backend_rbd: remove unnessary translated message marker Remove unnessary translated message marker _() for the VIR_WARN messages. 2016-09-26 Nitesh Konkar Fix various code comment typos 2016-09-26 John Ferlan qemu: Get/return compressedpath program Based upon a patch from Chen Hanxiao , rather than need to call virFindFileInPath twice, let's just save the path and pass it along with the compressed type. (NB: the second call would be in virExec as called from virCommandRunAsync which is called from qemuMigrationToFile using the argument 'compressor' which up to this point would be the string from the cfg file that isn't the fully qualified path). Since we now have the path, we can remove qemuCompressProgramName which would return NULL or the string representation of the compress type. 2016-09-26 John Ferlan qemu: Remove qemuCompressProgramAvailable There's only one caller and the code is duplicitous just converting the recently converted cfg image name back into it's string value in order to get/find the path to the image. A subsequent patch can return this path. 2016-09-26 John Ferlan qemu: Use qemuGetCompressionProgram for error paths Let's do some more code reuse - there are 3 other callers that care to check/get the compress program. Each of those though cares whether the requested cfg image is valid and exists. So, add a parameter to handle those cases. NB: We won't need to initialize the returned value in the case where the cfg image doesn't exist since the called program will handle that. 2016-09-26 John Ferlan qemu: Alter qemuGetCompressionProgram warning message Add a new parameter 'styleFormat' to be used when printing the warning message so that it's "clearer" what style of compression call caused the error. Add that style to both messages as a paremter. Also a VIR_WARN error message doesn't need to be translated (e.g. inside _()), so remove the need for the translation. 2016-09-26 John Ferlan qemu: Remove getCompressionType There's only one caller now anyway... Besides it's just a shell for getting the compress type. Subsequent patches will return the path to the compression program. qemu: Introduce helper qemuGetCompressionProgram Split out the guts of getCompressionType to perform the same functionality in the new helper program with a subsequent patch goal to be reusable for other callers making similar checks/calls to ensure the compression type is valid and that the compression program cannot be found. qemu: Adjust doCoreDump to call getCompressionType Rather than calling getCompressionType from each of the callers, just call it from doCoreDump. A subsequent patch will be adjust the code even more. qemu: Move getCompressionType A subsequent patch will adjust the 3 callers to just call from doCoreDump. 2016-09-26 Cédric Bosdonnat apparmor: move qemu-bridge-helper to libvirtd profile qemu-bridge-helper is only called from libvirtd, it has to be moved from the qemu domain abstraction to the usr.sbin.libvirtd profile. 2016-09-23 Cédric Bosdonnat libxl: increase usbdevice list only when finding such an input device If passing an empty usbdevice_list to libxl, qemu will always get an -usb parameter for HVM guests with only non-USB input devices. This causes qemu to crash when passing pvusb device on HVM guests. The solution is to allocate the list only when an item to put in it is found. 2016-09-23 Michal Privoznik qemuBuildHostNetStr: Realign 2016-09-23 Pavel Hrdina qemuhelptest: regenerate data for qemu-kvm-1.2.0 qemuhelptest: regenerate data for qemu-kvm-0.13.0 qemuhelptest: regenerate data for qemu-kvm-0.12.3 qemuhelptest: regenerate data for qemu-1.2.0 This patch also removes device data for qemu-1.2.0 as it was removed for qemu-kvm-1.2.0 by commit ae3e29e6e. They are not required because we parse only version from help output and return with error that this qemu is too new to use help parsing. qemuhelptest: regenerate data for qemu-1.1.0 qemuhelptest: regenerate data for qemu-1.0 qemuhelptest: regenerate data for qemu-0.12.1 qemuhelptest: remove downstream test data 2016-09-22 John Ferlan qemu: Fix improper indention Commit id 'ce61c164' indented wrong - not sure how I did that... 2016-09-22 Nitesh Konkar Fix Multiple Typos 2016-09-22 Jiri Denemark Move CMT feature filtering to QEMU driver It really doesn't belong to the generic CPU driver. 2016-09-22 Jiri Denemark qemu: Update guest CPU def in live XML Storing the updated CPU definition in the live domain definition saves us from having to update it over and over when we need it. Not to mention that we will soon further update the CPU definition according to QEMU once it's started. A highly wanted side effect of this patch, libvirt will pass all CPU features explicitly specified in domain XML to QEMU, even those that are already included in the host model. This patch should fix the following bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1207095 https://bugzilla.redhat.com/show_bug.cgi?id=1339680 https://bugzilla.redhat.com/show_bug.cgi?id=1371039 https://bugzilla.redhat.com/show_bug.cgi?id=1373849 https://bugzilla.redhat.com/show_bug.cgi?id=1375524 https://bugzilla.redhat.com/show_bug.cgi?id=1377913 2016-09-22 Jiri Denemark cpu: Rework cpuCompare* APIs Both cpuCompare* APIs are renamed to virCPUCompare*. And they should now work for any guest CPU definition, i.e., even for host-passthrough (trivial) and host-model CPUs. The implementation in x86 driver is enhanced to provide a hint about -noTSX Broadwell and Haswell models when appropriate. cpu: Document missing parameters for cpuCompare* cpu: Introduce virCPUCheckFeature The function is similar to virCPUDataCheckFeature, but it works directly on CPU definition rather than requiring it to be transformed into CPU data first. cpu: Rework virCPUDataCheckFeature To match our coding style and to provide better debug and error messages. cpu: Rename cpuHasFeature to virCPUDataCheckFeature cpu: Introduce virCPUTranslate The API is supposed to make sure the provided CPU definition does not use a CPU model which is not supported by the hypervisor (if at all possible, of course). cpu: Set nfeatures_max correctly in x86Decode Keeping nfeatures_max set to 0 while nfeatures > 0 and some features are already stored in features array is just asking for problems once we want to add a new feature into the array. 2016-09-22 Jiri Denemark cpu: Rework cpuUpdate The reworked API is now called virCPUUpdate and it should change the provided CPU definition into a one which can be consumed by the QEMU command line builder: - host-passthrough remains unchanged - host-model is turned into custom CPU with a model and features copied from host - custom CPU with minimum match is converted similarly to host-model - optional features are updated according to host's CPU 2016-09-22 Jiri Denemark cpu: Add x86FeatureInData The function checks CPUID data for a given feature. cpu: Report error for unknown features in x86HasFeature cpu: Make x86ModelFromCPU a bit smarter x86ModelFromCPU is used to provide CPUID data for features matching @policy. This patch allows callers to set @policy to -1 to get combined CPUID for all CPU features (including those implicitly provided a CPU model) specified in CPU def. cpu: Make x86ModelFromCPU easier to read qemu: Introduce virQEMUCapsIsCPUModeSupported qemu: Introduce virQEMUCapsGetHostModel 2016-09-22 Jiri Denemark Show host model in domain capabilities The domain capabilities XML is capable of showing whether each guest CPU mode is supported or not with a possibility to provide additional details. This patch enhances host-model capability to advertise the exact CPU model which will be used as a host-model: ... Broadwell Intel ... 2016-09-22 Jiri Denemark cpu: Drop false support for ARM cpu-model The ARM CPU driver wrongly reported host CPU model as "host", which made host-model to be just an alias for host-passthrough. Let's drop this insanity. 2016-09-22 Jiri Denemark qemu: Store host-model CPU in qemu capabilities Host capabilities provide libvirt's view of the host CPU, but for a useful support for host-model CPUs we really need a hypervisor's view of the CPU. And since the view can be differ with emulator, qemu capabilities is the best place to store the host CPU model. This patch just copies the CPU model from host capabilities, but this will change in the future. 2016-09-22 Jiri Denemark conf: Introduce virCPUDefCopyModelFilter The function filters all CPU features through a given callback while copying CPU model related parts of a CPU definition. conf: Introduce virCPUDefStealModel The function moves CPU model related parts from one CPU definition to another. It can be used to avoid unnecessary copies from a temporary CPU definitions which will be freed anyway. conf: Introduce virCPUDefCopyWithoutModel Useful for copying a CPU definition without model related parts (i.e., without model name, feature list, vendor). qemu: Propagate virCapsPtr to virQEMUCapsNewForBinaryInternal schema: Separate CPU related definitions into cputypes.rng domcaps: Add CPU usable flag In case a hypervisor is able to tell us a list of supported CPU models and whether each CPU models can be used on the current host, we can propagate this to domain capabilities. This is a better alternative to calling virConnectCompareCPU for each supported CPU model. domcaps: Show only CPU models supported by libvirt Listing all CPU models supported by QEMU in domain capabilities makes little sense when libvirt will refuse any model it doesn't know about. cpu: Don't overwrite errors in cpuGetModels cpuGetSubDriver already reports a useful error. cpu: Special case models == NULL in cpuGetModels Some CPU drivers (such as arm) do not provide list of CPUs libvirt supports and just pass any CPU model from domain XML directly to QEMU. Such driver need to return models == NULL and success from cpuGetModels. qemu: Fill in CPU domain capabilities qemu: Introduce virQEMUCapsGuestIsNative To have a single place where we decide whether a guest can run natively on a host. qemuxml2argvtest: Reorder CPU features The x86 CPU driver translated each CPU definition from domain XML into CPUID data and then back to CPU definition. This effectively sorted the list of CPU features according to their CPUID values. Since this is going to change, we need to reorder CPU features in a few test files to make sure the generated QEMU command lines will not change. qemuxml2argvtest: Set correct architecture for KVM guests Testing PPC64/AArch64 KVM domains on x86_64 host only works because we have a lot of bugs in our code. Since this series is going to fix them, we need to make sure the host architecture matches guest for KVM domains. qemuxml2argvtest: Properly setup CPU models in qemuCaps Adding x86 CPU models into a list of supported CPUs for non-x86 architectures is not a very good idea. Each architecture we test needs to maintain its own list of supported CPU models. qemu: Separate guest CPU validation from command line creation qemu_command.c should deal with translating our domain definition into a QEMU command line and nothing else. testutilsqemu: Add default CPU for PPC64 architectures testutilsqemu: Helpers for changing host CPU and arch Changing a host architecture or a CPU is not as easy as assigning a new value to the appropriate element in virCaps since there is a relation between the CPU and host architecture (we don't really want to test anything on an AArch64 host with core2duo CPU). This patch introduces qemuTestSetHostArch and qemuTestSetHostCPU helpers which will make sure the host architecture matches the host CPU. qemuxml2argvtest: Properly initialize qemuCaps->arch qemuCaps->arch should match the guest architecture from domain XML. qemuxml2argvtest: Update qemuCaps after parsing domain XML Some parts of qemuCaps depend on guest architecture, machine type, and possibly other things that we know only once the domain XML has been parsed. Let's move all these updates into a dedicated function. qemuxml2argvtest: Reorder functions testCompareXMLToArgv will soon need to call a few function which are defined further in the code. Let's move them up a bit. qemuxml2argvtest: Get rid of testCompareXMLToArgvHelper qemuxml2argvtest: Rename "out" labels as "cleanup" qemuxml2argvtest: Rename extraFlags as qemuCaps qemu: Use virDomainCapsCPUModels for cpuDefinitions The list of supported CPU models in domain capabilities is stored in virDomainCapsCPUModels. Let's use the same object for storing CPU models in QEMU capabilities. 2016-09-22 Jiri Denemark domcaps: Add support for listing supported CPU models The patch adds element to domain capabilities XML: Broadwell Broadwell-noTSX ... Applications can use it to inspect what CPU configuration modes are supported for a specific combination of domain type, emulator binary, guest architecture and machine type. 2016-09-22 Jiri Denemark cpuGetModels: Switch to virArch Our internal APIs mostly use virArch rather than strings. Switching cpuGetModels to virArch will save us from unnecessary conversions in the future. 2016-09-21 Jim Fehlig libxl: support VIR_MIGRATE_PERSIST_DEST migration flag By default, virt-manager (and likely other libvirt-based apps) sets the VIR_MIGRATE_PERSIST_DEST flag when invoking the migrate API, which fails in a Xen setup since the libxl driver does not support the flag. Persisting a domain is a trivial task in the grand scheme of migration, so be nice to libvirt apps and add support for VIR_MIGRATE_PERSIST_DEST in the libxl driver. 2016-09-21 Pavel Hrdina qemu_process: move graphics validation into separate function 2016-09-21 Chen Hanxiao qemu_process: show shutoff reasons when debug log disabled We have a few of senarios that libvirtd would invoke qemuProcessStop and leave a "shutting down" in /var/log/libvirt/qemu/$DOMAIN.log. The shutoff reason showing in debug log is also very important for us to know why VM shutting down in domain log, as we seldom enable debug log of libvirtd. 2016-09-21 Nitesh Konkar libvirt-nodedev.c:fix a typo libvirt-admin.c:fix a typo driver.c:fix a typo domain_conf.c:fix the comment. domain_conf.c:fix a typo domain_conf.c:fix a typo 2016-09-21 Michal Privoznik migration: Document we don't copy storage during offline migration https://bugzilla.redhat.com/show_bug.cgi?id=1322717 During offline migration, no storage is copied. Nor disks, nor NVRAM file, nor anything. We use qemu for that and because domain is not running there's nobody to copy that for us. We should document this to avoid confusing users. 2016-09-21 Peter Krempa qemu: driver: Don't return automatic NUMA emulator pinning data for persistentDef Calling virDomainGetEmulatorPinInfo on a live VM with automatic NUMA pinning and VIR_DOMAIN_AFFECT_CONFIG would return the automatic pinning data in some cases which is bogus. Use the autoCpuset property only when called on a live definition. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1365779 2016-09-21 Peter Krempa qemu: driver: Don't return automatic NUMA vCPU pinning data for persistentDef Calling virDomainGetVcpuPinInfo on a live VM with automatic NUMA pinning and VIR_DOMAIN_AFFECT_CONFIG would return the automatic pinning data in some cases which is bogus. Use the autoCpuset property only when called on a live definition. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1365779 2016-09-21 Peter Krempa qemu: domain: Add macro to simplify access to vm private data Sometimes adding a separate variable to access vm->privateData is not necessary. Add a macro that will do the typecasting rather than having to add a temp variable to force the compiler to typecast it. conf: Introduce virDomainObjGetOneDefState Return whether the live or persistent definition was returned. Sometimes it's necessary to base the decisions on this. 2016-09-21 Jiri Denemark qemu: Ignore graphics cookie if port == 0 Old libvirt represents as In this mode, QEMU doesn't listen for SPICE connection anywhere and clients have to use virDomainOpenGraphics* APIs to attach to the domain. That is, the client has to run on the same host where the domains runs and it's impossible to tell the client to reconnect to the destination QEMU during migration (unless there is some kind of proxy on the host). While current libvirt correctly ignores such graphics devices when creating graphics migration cookie, old libvirt just sends in the cookie. After seeing this cookie, we happily would call client_migrate_info QMP command and wait for SPICE_MIGRATE_COMPLETED event, which is quite pointless since the doesn't know where to connecti anyway. We should just ignore such cookies. https://bugzilla.redhat.com/show_bug.cgi?id=1376083 2016-09-21 Jiri Denemark qemuDomainOpenGraphics: Start job early Checking if a domain's definition or if it is active before we got a job is pointless since the domain might have changed in the meantime. Luckily libvirtd didn't crash when the API tried to talk to an inactive domain: debug : qemuDomainObjBeginJobInternal:2914 : Started job: modify (async=none vm=0x7f8f340140c0 name=ble) debug : qemuDomainObjEnterMonitorInternal:3137 : Entering monitor (mon=(nil) vm=0x7f8f340140c0 name=ble) warning : virObjectLock:319 : Object (nil) ((unknown)) is not a virObjectLockable instance debug : qemuMonitorOpenGraphics:3505 : protocol=spice fd=27 fdname=graphicsfd skipauth=1 error : qemuMonitorOpenGraphics:3508 : invalid argument: monitor must not be NULL debug : qemuDomainObjExitMonitorInternal:3160 : Exited monitor (mon=(nil) vm=0x7f8f340140c0 name=ble) debug : qemuDomainObjEndJob:3068 : Stopping job: modify (async=none vm=0x7f8f340140c0 name=ble) 2016-09-21 Nitesh Konkar stream.c: fix a typo Reviewed-by: Michal Privoznik config-post.h:fix a typo 2016-09-20 Nikolay Shirokovskiy qemu: agent: give better error messages whe agent monitor is down We can receive NULL as sync reply in two situations. First is garbage sync reply and this situation is handled by resending sync message. Second is different cases of rebooting guest, destroing domain etc and we can give more meaningful error message. Actually we have this error message in qemuAgentCommand already which checks for the same sitatuion. AFAIK case with mon->running is just to be safe on adding some future(?) cases of returning NULL reply. qemu: agent: reissue sync on garbage sync reply We can easily handle receiving garbage on sync. We don't have to make client deal with this situation. We just need to resend sync command but this time garbage is not be possible. qemu: agent: skip outdated sync replies When we wait for sync reply we can receive delayed reply to syncs or commands that were sent erlier. We can safely skip them until we receive sync reply with correct id. There is no much sense report this situation to client. Actually with a bit of "luck" if we involve client into this the play can go on forever: send sync 0, receive sync reply -1, send sync 1, receive reply 0 ... 2016-09-20 Nikolay Shirokovskiy qemu: agent: ignore garbage while waiting for sync reply After sync is sent we can receive garbare and this is not error. Consider next regular case: 1. libvirtd sends sync 2. qga sends partial sync reply and die 3. libvirtd sends sync 4. qga sends sync reply 5. libvirtd receives garbage (half of first reply and second reply together) We should handle this situation as it is recoverable. Next sync can succeed. Let's report reply is NULL, it will be converted to the VIR_ERR_AGENT_UNSYNCED which signals client to retry. 2016-09-20 Nikolay Shirokovskiy qemu: agent: ignore delayed reply when unsynced Errors in qemuAgentIOProcessLine stop agent IO processing just like any regular IO error, however some of current errors that this functions spawns are false positives. Consider next case for example: 1. send sync (unsynced state) 2. receive sync reply (sync established) 3. command send, but timeout occured (unsynced state) 4. receive command reply Last IO triggers error because current code ignores only delayed syncs when unsynced We should not treat any delayed reply as error in unsynced state. Until client and qga are not in sync delayed reply to any command is possible. msg == NULL is the exact criterion that we are not in sync. 2016-09-20 Martin Kletzander conf: Add some shmem helpers for future use qemu: Add qemuAssignDeviceShmemAlias and use it qemu: Rename qemuBuildShmemBackendStr to qemuBuildShmemBackendChrStr There will be more backends in the future so let's not complicate it. qemu: Abstract shmem socket path preparation Put it into qemuDomainPrepareShmemChardev() so it can be used later. Also don't fill in the path unless the server option is enabled. qemu: Reorder shmem params nicely Always format id first so that we don't need to do that twice in different code paths. qemu: Move common checks outside qemuBuildShmemDevLegacyStr Some checks will need to be performed for newer device types as well, so let's not duplicate them. qemu: Rename qemuBuildShmemDevStr to qemuBuildShmemDevLegacyStr This will make sense after adding support for newer device types. qemu: Make qemuBuildShmemDevStr static It isn't used anywhere else. 2016-09-20 Andrea Bolognani virtlogd: Don't stop or restart along with libvirtd Commit 839a060 tied the lifecycle of virtlogd more closely to that of libvirtd. Unfortunately, while starting virtlogd when libvirtd is started is definitely a good idea, restarting virtlogd or shutting it down at any time outside of system poweroff is not. Revert part of that commit by removing the PartOf= lines, meaning that only startup requests will be propagated from libvirtd to virtlogd. Resolves: https://bugzilla.redhat.com/1372576 2016-09-20 Erik Skultety virt-admin: Add some command aliases to provide syntax sugar over ugly commands Make use of the new recently introduced alias handling for virt-admin srv-* commands. virt-admin: Tweak command parsing logic so that aliases point to new commands Change the logic in a way, so that VSH_CMD_FLAG_ALIAS behaves similarly to how VSH_OT_ALIAS for command options, i.e. there is no need for code duplication for the alias and the aliased command structures. Along with that change, switch any existing VSH_CMD_FLAG_ALIAS occurrences to this new format. Also, since this patch introduces a new command structure element, adjust the virsh-self-test test to make sure we won't ever miss to specify the '.alias' member for an aliased command because doing that would lead to an internal error. virsh: man: Do not document our old deprecated nodedev-dettach command The command is deprecated due to being grammatically incorrect, but for backwards compatibility reasons cannot be removed. However, we should not document such commands. vsh: discard vshCmddefOptFill and move its body to vshCmddefOptParse Recent changes extracted the command internals validation routine from vshCmddefOptParse method which now just calls vshCmddefOptFill. Therefore, make vshCmddefOptFill the new vshCmddefOptParse and drop the unnecessary name. vsh: Extract vshCmddefCheckInternals from vshCmddefOptParse Originally introduced by commit 2432521e which correctly split vshCmddefOptParse into command's options validation and options parsing. However, command's 'internals' are not tied solely to .options, rather it should be about the overall structure, therefore the validation should be extracted from vshCmddefOptParse and performed only within our test suite, i.e. in vshSelfTest. vsh: vshCmddefHelp: Drop the unnecessary 'else' branch If the initial check is true the function immediately returns so there's no need to enclose the code following the check within an 'else' block. Also, by removing the 'else' block, the declarations need to be moved to beginning of the function block to conform with our guidelines. vsh: vshCmddefHelp: Drop unnecessary variable 'help' Since it's used on a single place only, it can easily be replaced by the right side of the original assignment. vsh: Enforce checking for missing help string The intention is to move vshCmddefCheckInternals out of vshCmddefOptParse to our test suite. First step to do that is to enforce checking for an existing help string (that also means it's non-empty) in a command because a command without a help is not much of a use. 2016-09-20 Daniel P. Berrange Add missing 'extern' for VIR_EXPORT_VAR macro The recent update to gnulib commit 9d7a37ecb2ab516c19164a57b8aeb32016a99d2c Author: Eric Blake Date: Thu Sep 15 15:12:52 2016 -0500 build: update to latest gnulib Pulled in a change that adds -fno-common to the default compiler flags commit bf8e658ffadb95d444f56d222d04c9af955af765 Author: Jim Meyering Date: Fri Sep 2 09:16:16 2016 -0700 manywarnings: add -fno-common This caused libvirt Mingw build to break with the compiler reporting 100's of definitions of virConnectAuthPtrDefault ./.libs/libvirt_util.a(libvirt_util_la-virarch.o):virarch.c:(.bss+0x0): multiple definition of `virConnectAuthPtrDefault' ./.libs/libvirt_util.a(libvirt_util_la-viralloc.o):viralloc.c:(.bss+0x0): first defined here ./.libs/libvirt_util.a(libvirt_util_la-viratomic.o):viratomic.c:(.bss+0x0): multiple definition of `virConnectAuthPtrDefault' ./.libs/libvirt_util.a(libvirt_util_la-viralloc.o):/home/berrange/src/virt/libvirt/src/util/viralloc.c:87: first defined here ./.libs/libvirt_util.a(libvirt_util_la-viraudit.o):viraudit.c:(.bss+0x0): multiple definition of `virConnectAuthPtrDefault' ./.libs/libvirt_util.a(libvirt_util_la-viralloc.o):/home/berrange/src/virt/libvirt/src/util/viralloc.c:87: first defined here ./.libs/libvirt_util.a(libvirt_util_la-virauth.o):virauth.c:(.bss+0x0): multiple definition of `virConnectAuthPtrDefault' ./.libs/libvirt_util.a(libvirt_util_la-viralloc.o):/home/berrange/src/virt/libvirt/src/util/viralloc.c:87: first defined here ./.libs/libvirt_util.a(libvirt_util_la-virauthconfig.o):virauthconfig.c:(.bss+0x0): multiple definition of `virConnectAuthPtrDefault' ...snip... The cause is our VIR_EXPORT_VAR macro which has some magic on win to add dllexport/dllimport to the variable declaration. Unfortunately the dllexport branch missed off the 'extern' keyword, so the header file was in fact declaring an instance of the variable in every source file. Previously the linker would merge all these definitions into one, but that no longer happens due to -fno-common 2016-09-20 Pavel Hrdina test: update capability replies for qemu-2.7.0 2016-09-20 Chen Hanxiao qemu_process: fix a typo s/unitl/until 2016-09-19 Laine Stump tests: fix use of fixedcontent variable Commit 8563560026d192c2cf047b550ffd468692245ed6 switched from hardcoded use of strcontent to hardcoded use of fixedcontent (fixedcontent is *sometimes* a copy of strcontent with a \n appended). This was a problem because sometimes fixedcontent is *not* a copy of strcontent, but is instead NULL, leading to the regenerated test case output being a 0 length file. This patch creates a new const char *cmpcontent initialized to strcontent, but changed to fixedcontent if/when fixedcontent is created, then always uses cmpcontent instead of (str|fixed)content. 2016-09-19 Eric Blake Revert "configure: Check for major() more strictly" This reverts commit d53fa838e18d8c192296487ff40c3a37420b6d06, which is no longer needed now that gnulib does the job. build: update to latest gnulib In particular, pull in gnulib's fix for the broken AC_HEADER_MAJOR macro in autoconf. 2016-09-19 Michal Privoznik qemu: Introduce qemuGetHupageMemPath Now that we have two same implementations for getting path for huge pages backed guest memory, lets merge them into one function. 2016-09-19 Michal Privoznik qemuBuildMemoryBackendStr: Don't crash if no hugetlbfs is mounted When trying to migrate a huge page enabled guest, I've noticed the following crash. Apparently, if no specific hugepages are requested: and there are no hugepages configured on the destination, we try to dereference a NULL pointer. Program received signal SIGSEGV, Segmentation fault. 0x00007fcc907fb20e in qemuGetHugepagePath (hugepage=0x0) at qemu/qemu_conf.c:1447 1447 if (virAsprintf(&ret, "%s/libvirt/qemu", hugepage->mnt_dir) < 0) (gdb) bt #0 0x00007fcc907fb20e in qemuGetHugepagePath (hugepage=0x0) at qemu/qemu_conf.c:1447 #1 0x00007fcc907fb2f5 in qemuGetDefaultHugepath (hugetlbfs=0x0, nhugetlbfs=0) at qemu/qemu_conf.c:1466 #2 0x00007fcc907b4afa in qemuBuildMemoryBackendStr (size=4194304, pagesize=0, guestNode=0, userNodeset=0x0, autoNodeset=0x0, def=0x7fcc70019070, qemuCaps=0x7fcc70004000, cfg=0x7fcc5c011800, backendType=0x7fcc95087228, backendProps=0x7fcc95087218, force=false) at qemu/qemu_command.c:3297 #3 0x00007fcc907b4f91 in qemuBuildMemoryCellBackendStr (def=0x7fcc70019070, qemuCaps=0x7fcc70004000, cfg=0x7fcc5c011800, cell=0, auto_nodeset=0x0, backendStr=0x7fcc70020360) at qemu/qemu_command.c:3413 #4 0x00007fcc907c0406 in qemuBuildNumaArgStr (cfg=0x7fcc5c011800, def=0x7fcc70019070, cmd=0x7fcc700040c0, qemuCaps=0x7fcc70004000, auto_nodeset=0x0) at qemu/qemu_command.c:7470 #5 0x00007fcc907c5fdf in qemuBuildCommandLine (driver=0x7fcc5c07b8a0, logManager=0x7fcc70003c00, def=0x7fcc70019070, monitor_chr=0x7fcc70004bb0, monitor_json=true, qemuCaps=0x7fcc70004000, migrateURI=0x7fcc700199c0 "defer", snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START, standalone=false, enableFips=false, nodeset=0x0, nnicindexes=0x7fcc95087498, nicindexes=0x7fcc950874a0, domainLibDir=0x7fcc700047c0 "/var/lib/libvirt/qemu/domain-1-fedora") at qemu/qemu_command.c:9547 2016-09-19 Chen Hanxiao qemu_agent|monitor: use different log on hangup event Both qemu monitor and agent print the same log on HUANGUP event, which would be confusing when reading libvirtd log. This patch will give a different log message to them. Reviewed-by: Michal Privoznik 2016-09-19 Daniel P. Berrange docs: expand docs on user x509 cert locations The layout in $HOME/.pki is different from that in /etc/pki but we never tell anyone about this trap. Add docs showing the required $HOME/.pki layout. 2016-09-16 Martin Kletzander conf: Allow copying of shmem defs This way we'll be able to hotplug with both --live and --config in one API call. schema: Allow shmem to have alias It already is used and tests will be automatically added in later patches. qemuhotplugtest: Only read result_filename if used 2016-09-16 Laszlo Ersek qemu: map "virtio" video model to "virt" machtype correctly (arm/aarch64) Most of QEMU's PCI display device models, such as: libvirt video/model/@type QEMU -device ------------------------- ------------ cirrus cirrus-vga vga VGA qxl qxl-vga virtio virtio-vga come with a linear framebuffer (sometimes called "VGA compatibility framebuffer"). This linear framebuffer lives in one of the PCI device's MMIO BARs, and allows guest code (primarily: firmware drivers, and non-accelerated OS drivers) to display graphics with direct memory access. Due to architectural reasons on aarch64/KVM hosts, this kind of framebuffer doesn't / can't work in qemu-system-(arm|aarch64) -M virt machines. Cache coherency issues guarantee a corrupted / unusable display. The problem has been researched by several people, including kvm-arm maintainers, and it's been decided that the best way (practically the only way) to have boot time graphics for such guests is to consolidate on QEMU's "virtio-gpu-pci" device. >From , libvirt supports but libvirt unconditionally maps @type='virtio' to QEMU's "virtio-vga" device model. (See the qemuBuildDeviceVideoStr() function and the "qemuDeviceVideo" enum impl.) According to the above, this is not right for the "virt" machine type; the qemu-system-(arm|aarch64) binaries don't even recognize the "virtio-vga" device model (justifiedly). Whereas "virtio-gpu-pci", which is a pure virtio device without a compatibility framebuffer, is available, and works fine. (The ArmVirtQemu ("AAVMF") platform of edk2 -- that is, the UEFI firmware for "virt" -- supports "virtio-gpu-pci", as of upstream commit 3ef3209d3028. See .) Override the default mapping of "virtio", from "virtio-vga" to "virtio-gpu-pci", if qemuDomainMachineIsVirt() evaluates to true. Cc: Andrea Bolognani Cc: Drew Jones Cc: Marc-André Lureau Cc: Martin Kletzander Suggested-by: Marc-André Lureau Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1372901 Acked-by: Martin Kletzander 2016-09-16 Martin Kletzander util: Expose virHostCPUStatsAssign on non-Linux There is nothing Linux-specific in that function. Also since commit 8c3b5bf48123783b812b97360db7ac51f1889e17 mingw build is broken due to the fact that this function is not compiled in the library. 2016-09-15 Tomáš Ryšavý test driver: Implement testNodeGetFreePages. test driver: added pages sizes into XML into test driver. test driver: Implement testNodeGetFreeMemory test driver: Implement testNodeGetCPUStats test driver: Implement virConnectGetType. test driver: Implement virConnectGetSysinfo. test_driver: Replace magic constant In testOpenDefault we create a virtual computer that is later presented to user. We also pretend to have NUMA cells and initialize them somehow. But whilst doing so a magical constant is used. Drop it. virhostcpu: Expose virHostCPUStatsAssign We will need this function shortly when implementing nodeGetCPUStats in the test driver. 2016-09-15 Shivaprasad G Bhat virsh: use virConnectGetDomainCapabilities with maxvcpus virsh maxvcpus --type kvm output is useless on PPC. Also, in commit e6806d79 we documented not rely on virConnectGetMaxVcpus output. Fix the maxvcpus to use virConnectGetDomainCapabilities now to make it useful. The call is made to use the default emulator binary and to check for the host machine and arch which is what the command intends to show anyway. 2016-09-15 Martin Kletzander tests: Use fixedcontent variable Commit ca32929908bbc94116493ad8915e7cd7ae0f57d5 added function virTestCompareToFile(), but forgot to use a fixedcontent value for the actual comparison. That lead to VIR_TEST_DEBUG=1 showing (for some tests) all the actual output from the first error to the end of the string due to the difference being an endline in the end. 2016-09-15 Jason Miesionczek hyperv: additional server 2008 wmi classes 2016-09-14 Michal Privoznik tests: Self test virt-admin Just like we are running 'virsh self-test' from within our test suite, we should run 'virt-admin self-test' too. virsh: Move cmdSelfTest to vsh This command should be exposed to other shells of ours. They are gonna need it as soon as we want to test them too. 2016-09-14 Peter Krempa qemu: domain: Don't infer vcpu state Use the state information (online, hotpluggable) provided by the monitor code rather than trying to infer it. This fixes an issue where on architectures that require hotplug of multiple threads at once the sub-cores would get updated as offline on daemon restart thus creating an invalid configuration. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1375783 2016-09-14 Peter Krempa qemu: monitor: Add vcpu state information to monitor data Return whether a vcpu entry is hotpluggable or online so that upper layers don't have to infer the information from other data. Advantage is that this code can be tested by unit tests. 2016-09-14 Peter Krempa qemu: monitor: qemuMonitorGetCPUInfoHotplug: Add iterator 'anycpu' Add separate iterator for iterating all the entries qemu: monitor: Use a more obvious iterator name The algorithm that matches data from query-cpus and query-hotpluggable-cpus is quite complex. Start using descriptive iterator names to avoid confusion. 2016-09-14 Michal Privoznik virDomainGetBlockJobInfo: Fix corner case when qemu reports no info https://bugzilla.redhat.com/show_bug.cgi?id=1372613 Apparently, some management applications use the following code pattern when waiting for a block job to finish: while (1) { virDomainGetBlockJobInfo(dom, disk, info, flags); if (info.cur == info.end) break; sleep(1); } Problem with this approach is in its corner cases. In case of QEMU, libvirt merely pass what has been reported on the monitor. However, if the block job hasn't started yet, qemu reports cur == end == 0 which tricks mgmt apps into thinking job is complete. The solution is to mangle cur/end values as described here [1]. 1: https://www.redhat.com/archives/libvir-list/2016-September/msg00017.html 2016-09-14 Michal Privoznik qemuDomainGetBlockJobInfo: Move info translation into separate func Even though we merely just pass to users whatever qemu provided on the monitor, we still do some translation. For instance we turn bytes into mebibytes, or fix job type if needed. However, in the future there is more fixing to be done so this code deserves its own function. 2016-09-14 Daniel P. Berrange tests: use a fixed chardev TLS path The test qemuxml2argv-serial-tcp-tlsx509-chardev.args will fail if libvirt is built with a --sysconfdir arg that is not /etc. Fix this by setting a hardcoded path in the test code. 2016-09-14 Peter Krempa numa: Rename virNumaGetHostNodeset and make it return only nodes with memory Name it virNumaGetHostMemoryNodeset and return only NUMA nodes which have memory installed. This is necessary as the kernel is not very happy to set the memory cgroup setting for nodes which do not have any memory. This would break vcpu hotplug with following message on such configruation: Invalid value '0,8' for 'cpuset.mems': Invalid argument Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1375268 2016-09-14 Peter Krempa util: numa: Remove impossible error handling The code guarantees that virBitmapSetBit won't be called with out of range values. Just ignore the return value and remove dead error handling. 2016-09-13 Michal Privoznik configure: Check for major() more strictly Thing is, in f3f15cc24 I'm trying to adapt libvirt to the newest glibc where major()/minor()/makedev() are moved from sys/types.h to sys/sysmacros.h. However, my commit back then expect autoconf to be fixed too as we already use AC_HEADER_MAJOR to determine which header file the functions are in, but because the header files just trigger a warning and not a compile error, the autoconf macro detects the bad header file. This is just a workaround until autoconf macro is fixed. 2016-09-13 Erik Skultety tests: fix incorrect status handling by virsh-self-test The virsh-self-test script compared the test's return code with 1 and only if the return code matched this value then the test was marked as failed. Problem is that SIGSEGV returns 139 (or 11 to be precise, since shell reserves the MSB for abnormal exit signaling) which passes the check just fine and test then appears as successful which it most certainly wasn't. Therefore, flip the logic to compare against 0 instead and every other result will be treated as a failed test case. 2016-09-13 Peter Krempa qemu: Add missing 'p' to qemuCgrouEmulatorAllNodesRestore 2016-09-13 Nikolay Shirokovskiy virsh: add --keep-nvram option to undefine command qemu: add VIR_DOMAIN_UNDEFINE_KEEP_NVRAM support api: add VIR_DOMAIN_UNDEFINE_KEEP_NVRAM flag This flags specifies to keep nvram file if it is existed for inactive domains. 2016-09-12 Laine Stump conf: allow hotplugging "legacy PCI" device to manually addressed PCIe slot In a full domain config, libvirt allows overriding the normal PCI vs. PCI Express rules when a device address is explicitly provided (so, e.g., you can force a legacy PCI device to plug into a PCIe port, although libvirt would never do that on its own). However, due to a bug libvirt doesn't give this same leeway when hotplugging devices. On top of that, current libvirt assumes that *all* devices are legacy PCI. The result of all this is that it's impossible to hotplug a device into a PCIe port, even if you manually add the PCI address. This can all be traced to the function virDomainPCIAddressEnsureAddr(), and the fact that it calls virDomainPCIaddressReserveSlot() for manually set addresses, and that function hardcodes the argument "fromConfig" to false (meaning "this address was auto-assigned, so it should be subject to stricter validation"). Since virDomainPCIAddressReserveSlot() is just a one line simple wrapper around virDomainPCIAddressReserveAddr() (adding in a hardcoded reserveEntireSlot = true and fromConfig = false), all that's needed to solve the problem with no unwanted side effects is to replace that call for virDomainPCIAddressReserveSlot() with a direct call to virDomainPCIAddressReserveAddr(), but with reserveEntireSlot = true, fromConfig = true. That's what this patch does. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1337490 2016-09-12 Laine Stump qemu: fix improper initialization of cgroupControllers bitmap virQEMUDriverConfigNew() always initializes the bitmap in its cgroupControllers member to -1 (i.e. all 1's). Prior to commit a9331394, if qemu.conf had a line with "cgroup_controllers", cgroupControllers would get reset to 0 before going through a loop setting a bit for each named cgroup controller. commit a9331394 left out the "reset to 0" part, so cgroupControllers would always be -1; if you didn't want a controller included, there was no longer a way to make that happen. This was discovered by users who were using qemu commandline passthrough to use the "input-linux" method of directing keyboard/mouse input to a virtual machine: https://www.redhat.com/archives/vfio-users/2016-April/msg00105.html Here's the first report I found of the problem encountered after upgrading libvirt beyond v2.0.0: https://www.redhat.com/archives/vfio-users/2016-August/msg00053.html Thanks to sL1pKn07 SpinFlo for bringing the problem up in IRC, and then taking the time to do a git bisect and find the patch that started the problem. 2016-09-12 Martin Kletzander audit: Audit information about shmem devices Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1218603 2016-09-12 Daniel P. Berrange qemu: avoid parameter named 'listen' previous commit: commit 2c3223785c121eafbdd503f11549d7964461934f Author: John Ferlan Date: Mon Jun 13 12:30:34 2016 -0400 qemu: Add the ability to hotplug the TLS X.509 environment added a parameter "bool listen" in some methods. This unfortunately clashes with the listen() method, causing compile failures on certain platforms (RHEL-6 for example) 2016-09-12 John Ferlan storage: Need to properly read the crypt offset value Commit id 'a48c7141' altered how to determine if a volume was encrypted by adding a peek at an offset into the file at a specific buffer location. Unfortunately, all that was compared was the first "char" of the buffer against the expect "int" value. Restore the virReadBufInt32BE to get the complete field in order to compare against the expected value from the qcow2EncryptionInfo or qcow1EncryptionInfo "modeValue" field. This restores the capability to create a volume with encryption, then refresh the pool, and still find the encryption for the volume. 2016-09-12 John Ferlan storage: Need to refresh secret for luks volume after volume refresh A LUKS volume uses the volume secret type just like the QCOW2 secret, so adjust the loading of the default secrets to handle any volume that the virStorageFileGetMetadataFromBuf code has deemed to be an encrypted volume to search for the volume's secret. This lookup is done by volume usage where the usage is expected to be the path to volume. 2016-09-12 Nikolay Shirokovskiy log: Fix reporting OOM error incorrectly when defining a logging filter When a new filter is being defined, the return code is not handled properly, thus triggering OOM error reporting routine (bug introduced by 51b2606f). 2016-09-12 Jiri Denemark qemu: Don't use query-migrate on destination When migration fails, we need to poke QEMU monitor to check for a reason of the failure. We did this using query-migrate QMP command, which is not supposed to return any meaningful result on the destination side. Thus if the monitor was still functional when we detected the migration failure, parsing the answer from query-migrate always failed with the following error message: "info migration reply was missing return status" This irrelevant message was then used as the reason for the migration failure replacing any message we might have had. Let's use harmless query-status for poking the monitor to make sure we only get an error if the monitor connection is broken. https://bugzilla.redhat.com/show_bug.cgi?id=1374613 2016-09-12 John Ferlan util: Quiet the logging if perf file doesn't exist Commit id 'b00d7f29' shifted the opening of the /sys/devices/intel_cqm/type file from event enable to perf event initialization. If the file did not exist, then an error would be written to the domain log: 2016-09-06 20:51:21.677+0000: 7310: error : virFileReadAll:1360 : Failed to open file '/sys/devices/intel_cqm/type': No such file or directory Since the error is now handled in virPerfEventEnable by checking if the event_attr->attrType == 0 for CMT, MBML, and MBMT events - we can just use the Quiet API in order to not log the error we're going to throw away. Additionally, rather than using virReportSystemError, use virReportError and VIR_ERR_ARGUMENT_UNSUPPORTED in order to signify that support isn't there for that type of perf event - adjust the error message as well. 2016-09-12 Joao Martins libxl: implement virConnectBaselineCPU Akin to previous commit but for "virsh cpu-baseline" which computes a baseline CPU for a set of host cpu elements. libxl: implement virConnectCompareCPU Implement support for "virsh cpu-compare" so that we can calculate common cpu element between a pool of hosts, which had a requirement of providing host cpu description. 2016-09-12 Joao Martins libxl: describe host cpu features based on hwcaps Parse libxl_hwcap accounting for versions since Xen 4.4 - Xen 4.7. libxl_hwcaps is a set of cpuid leaves output that is described in [0] or [1] in Xen 4.7. This is a collection of CPUID leaves that we version in libvirt whenever feature words are reordered or added. Thus we keep the common ones in one struct and others for each version. Since libxl_hwcaps doesn't appear to have a stable format across all supported versions thus we need to keep track of changes as a compromise until it's exported in xen libxl API. We don't fail in initializing the driver in case parsing of hwcaps failed for that reason. In addition, change the notation on PAE feature such that is easier to read which bit it corresponds. [0] xen/include/asm-x86/cpufeature.h [1] xen/include/public/arch-x86/cpufeatureset.h 2016-09-12 Joao Martins libxl: describe host topology in capabilities Add support for describing cpu topology in host cpu element. In doing so, refactor hwcaps part to its own helper namely libxlCapsInitCPU to handle all host cpu related operations, including topology. 2016-09-12 Peter Krempa qemu: hotplug: Don't wait if cdrom tray is opened forcibly Qemu always opens the tray if forced to. Skip the waiting step in such case. This also helps if qemu does not report the tray change event when opening the cdrom forcibly (the documentation says that the event will not be sent although qemu in fact does trigger it even if @force is selceted). This is a workaround for a qemu issue where qemu does not send the tray change event in some cases (after migration with empty closed locked drive) and thus renders the cdrom useless from libvirt's point of view. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1368368 2016-09-12 Peter Krempa qemu: domain: Clear startup policy for dropped removable media When a source image is dropped when missing due to startup policy the policy needs to be cleared since it was relevant only for the given storage source. New sources need to update it if needed. 2016-09-09 Michal Privoznik qemu: Implement virtio-net rx_queue_size qemu_capabilities: Introduce virtio-net-*.rx_queue_size Just like in the previous commit, teach qemu driver to detect whether qemu supports this configuration knob or not. 2016-09-09 Michal Privoznik conf: Add support for virtio-net.rx_queue_size https://bugzilla.redhat.com/show_bug.cgi?id=1366989 QEMU added another virtio-net tunable [1]. It basically allows users to set the size of RX virtio ring. But because virtio-net uses two separate ring buffers to pass data from/to guest they named it explicitly rx_queue_size. We should expose it in our XML too. 1: http://lists.nongnu.org/archive/html/qemu-devel/2016-08/msg02029.html 2016-09-09 John Ferlan conf: Add new secret type "tls" Add a new secret usage type known as "tls" - it will handle adding the secret objects for various TLS objects that need to provide some sort of passphrase in order to access the credentials. The format is: Sample TLS secret mumblyfratz Once defined and a passphrase set, future patches will allow the UUID to be set in the qemu.conf file and thus used as a secret for various TLS options such as a chardev serial TCP connection, a NBD client/server connection, and migration. 2016-09-09 John Ferlan qemu: Add the ability to hotplug the TLS X.509 environment If the incoming XML defined a path to a TLS X.509 certificate environment, add the necessary 'tls-creds-x509' object to the VIR_DOMAIN_CHR_TYPE_TCP character device. Likewise, if the environment exists the hot unplug needs adjustment as well. Note that all the return ret were changed to goto cleanup since the cfg needs to be unref'd 2016-09-09 John Ferlan qemu: Add support for TLS X.509 path to TCP chardev backend When building a chardev device string for tcp, add the necessary pieces to access provide the TLS X.509 path to qemu. This includes generating the 'tls-creds-x509' object and then adding the 'tls-creds' parameter to the VIR_DOMAIN_CHR_TYPE_TCP command line. Finally add the tests for the qemu command line. This test will make use of the "new(ish)" /etc/pki/qemu setting for a TLS certificate environment by *not* "resetting" the chardevTLSx509certdir prior to running the test. Also use the default "verify" option (which is "no"). 2016-09-09 John Ferlan conf: Introduce chartcp_tls_x509_cert_dir Add a new TLS X.509 certificate type - "chardev". This will handle the creation of a TLS certificate capability (and possibly repository) for properly configured character device TCP backends. Unlike the vnc and spice there is no "listen" or "passwd" associated. The credentials eventually will be handled via a libvirt secret provided to a specific backend. Make use of the default verify option as well. 2016-09-09 John Ferlan conf: Add new default TLS X.509 certificate default directory Rather than specify perhaps multiple TLS X.509 certificate directories, let's create a "default" directory which can then be used if the service (e.g. for now vnc and spice) does not supply a default directory. Since the default for vnc and spice may have existed before without being supplied, the default check will first check if the service specific path exists and if so, set the cfg entry to that; otherwise, the default will be set to the (now) new defaultTLSx509certdir. Additionally add a "default_tls_x509_verify" entry which can also be used to force the peer verification option (for vnc it's a x509verify option). Add/alter the macro for the option being found in the config file to accept the default value. 2016-09-08 Jiri Denemark qemu: Remove stale transient def when migration fails If a migration of a domain which is already defined on the destination host failed early (before we tried to start QEMU), we would forget to remove the incoming transient definition. Later on when someone starts the domain on the destination host, we will use the stale incoming definition and the persistent definition will just be ignored. https://bugzilla.redhat.com/show_bug.cgi?id=1368774 2016-09-08 Jiri Denemark Add helper for removing transient definition The code for replacing domain's transient definition with the persistent one is repeated in several places and we'll need to add one more. Let's make a nice helper for it. 2016-09-08 Erik Skultety virt-admin: Fix some English grammar in the manpage virt-admin: Document srv-list command in virt-admin's man page srv-list command was missing in the man page. 2016-09-08 Christophe Fergeau virsh: Fix *-event error string When using virsh net-event non-existing-net the error message says that 'either --list or event type is required' This is misleading as 'virsh net-event $valid-event-type' is not going to work either. What is expected is 'virsh net-event --event $valid-event-type' This commit fixes the string in pool-event, nodedev-event, event, and net-event. 2016-09-08 Julio Faracco security: Fixing wrong label in virt-aa-helper.c. There is an issue with a wrong label inside vah_add_path(). The compilation fails with the error: make[3]: Entering directory '/tmp/libvirt/src' CC security/virt_aa_helper-virt-aa-helper.o security/virt-aa-helper.c: In function 'vah_add_path': security/virt-aa-helper.c:769:9: error: label 'clean' used but not defined goto clean; This patch moves 'clean' label to 'cleanup' label. 2016-09-07 Rufo Dogav Avoid segfault in virt-aa-helper when handling read-only filesystems This patch fixes a segfault in virt-aa-helper caused by attempting to modify a static string literal. It is triggered when a domain has a with type='mount' configured read-only and libvirt is using the AppArmor security driver for sVirt confinement. An "R" is passed into the function and converted to 'r'. 2016-09-07 Yanqiu Zhang storage: Delete extra wrap after vol-resize error This patch is to delete the extra wrap "\n" after failed vol-resize error for both "Failed to change size of volume to" and "Failed to change size of volume by". For error with wrap, there will be an extra wrap between two errors, such as: (1)# virsh vol-resize --pool default --vol vol-test 5M error: Failed to change size of volume 'vol-test' to 5M error: invalid argument: Can't shrink capacity below current capacity unless shrink flag explicitly specified (2)# virsh vol-resize /var/lib/libvirt/images/volds --shrink --delta 10M error: Failed to change size of volume 'volds' by 10M error: invalid argument: can't shrink capacity below existing allocation 2016-09-07 Peter Krempa qemu: driver: Remove unnecessary condition At this point it's guaranteed that 'persistentDef' is non-NULL so we don't need to check it again. 2016-09-07 Peter Krempa qemu: process: Fix start with unpluggable vcpus with NUMA pinning Similarly to vcpu hotplug the emulator thread cgroup numa mapping needs to be relaxed while hot-adding vcpus so that the threads can allocate data in the DMA zone. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1370084 2016-09-07 Peter Krempa qemu: cgroup: Extract temporary relaxing of cgroup setting for vcpu hotplug When hot-adding vcpus qemu needs to allocate some structures in the DMA zone which may be outside of the numa pinning. Extract the code doing this in a set of helpers so that it can be reused. 2016-09-07 Erik Skultety virt-admin: Output srv-clients-set data as unsigned int rather than signed Unfortunately, commit a8962f70 only fixed first half of the reported issue of virt-admin outputting negative values where unsigned int is expected by BZ below, so this commit represents the other missing half of the fix. resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1356769 2016-09-07 Maxim Nestratov util: fix crash in virClassIsDerivedFrom for CloseCallbacks objects There is a possibility that qemu driver frees by unreferencing its closeCallbacks pointer as it has the only reference to the object, while in fact not all users of CloseCallbacks called thier virCloseCallbacksUnset. Backtrace is the following: Thread #1: 0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 1 in virCondWait (c=, m=) at util/virthread.c:154 2 in virThreadPoolFree (pool=0x7f0810110b50) at util/virthreadpool.c:266 3 in qemuStateCleanup () at qemu/qemu_driver.c:1116 4 in virStateCleanup () at libvirt.c:808 5 in main (argc=, argv=) at libvirtd.c:1660 Thread #2: 0 in virClassIsDerivedFrom (klass=0xdeadbeef, parent=0x7f0837c694d0) at util/virobject.c:169 1 in virObjectIsClass (anyobj=anyobj@entry=0x7f08101d4760, klass=) at util/virobject.c:365 2 in virObjectLock (anyobj=0x7f08101d4760) at util/virobject.c:317 3 in virCloseCallbacksUnset (closeCallbacks=0x7f08101d4760, vm=vm@entry=0x7f08101d47b0, cb=cb@entry=0x7f081d078fc0 ) at util/virclosecallbacks.c:163 4 in qemuProcessAutoDestroyRemove (driver=driver@entry=0x7f081018be50, vm=vm@entry=0x7f08101d47b0) at qemu/qemu_process.c:6368 5 in qemuProcessStop (driver=driver@entry=0x7f081018be50, vm=vm@entry=0x7f08101d47b0, reason=reason@entry=VIR_DOMAIN_SHUTOFF_SHUTDOWN, asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_NONE, flags=flags@entry=0) at qemu/qemu_process.c:5854 6 in processMonitorEOFEvent (vm=0x7f08101d47b0, driver=0x7f081018be50) at qemu/qemu_driver.c:4585 7 qemuProcessEventHandler (data=, opaque=0x7f081018be50) at qemu/qemu_driver.c:4629 8 in virThreadPoolWorker (opaque=opaque@entry=0x7f0837c4f820) at util/virthreadpool.c:145 9 in virThreadHelper (data=) at util/virthread.c:206 10 in start_thread () from /lib64/libpthread.so.0 Let's reference CloseCallbacks object in virCloseCallbacksSet and unreference in virCloseCallbacksUnset. 2016-09-07 Yuri Pudgorodskiy qemu: guest agent: introduce new error code VIR_ERR_AGENT_UNSYNCED A separate error code will help recognize real failures from necessity to try again 2016-09-06 Michal Privoznik Make sure sys/types.h is included after sys/sysmacros.h In the latest glibc, major() and minor() functions are marked as deprecated (glibc commit dbab6577): CC util/libvirt_util_la-vircgroup.lo util/vircgroup.c: In function 'virCgroupGetBlockDevString': util/vircgroup.c:768:5: error: '__major_from_sys_types' is deprecated: In the GNU C Library, `major' is defined by . For historical compatibility, it is currently defined by as well, but we plan to remove this soon. To use `major', include directly. If you did not intend to use a system-defined macro `major', you should #undef it after including . [-Werror=deprecated-declarations] if (virAsprintf(&ret, "%d:%d ", major(sb.st_rdev), minor(sb.st_rdev)) < 0) ^~ In file included from /usr/include/features.h:397:0, from /usr/include/bits/libc-header-start.h:33, from /usr/include/stdio.h:28, from ../gnulib/lib/stdio.h:43, from util/vircgroup.c:26: /usr/include/sys/sysmacros.h:87:1: note: declared here __SYSMACROS_DEFINE_MAJOR (__SYSMACROS_FST_IMPL_TEMPL) ^ Moreover, in the glibc commit, there's suggestion to keep ordering of including of header files as implemented here. 2016-09-06 Nishith Shah tools: Pass opaque data in vshCompleter and introduce autoCompleteOpaque This patch changes the signature of vshCompleters, allowing to pass along some data that we might want to along with the completers; for example, we might want to pass the autocomplete vshControl along with the completer, in case the completer requires a connection to libvirtd. 2016-09-06 Roman Bogorodskiy util: hostcpu: improve CPU freq code for FreeBSD Current implementation uses the dev.cpu.0.freq sysctl that is provided by the cpufreq(4) framework and returns the actual CPU frequency. However, there are environments where it's not available, e.g. when running nested in KVM. In this case fall back to hw.clockrate that reports CPU frequency at the boot time. Resolves (hopefully): https://bugzilla.redhat.com/show_bug.cgi?id=1369964 2016-09-06 Andrea Bolognani libvirt-guests.service: Add Requires=libvirtd.service Having After=libvirtd.service merely ensures that, if both services are asked to start, libvirtd.service will start first. What we really want is for libvirtd.service to be started whenever libvirt-guests.service is asked to start. Adding a Requires= relationship guarantees that will happen. 2016-09-06 Andrea Bolognani libvirt-guests.service: Split After= relationship We use a separate line for each After= relationship in other unit files: do the same here for consistency's sake, and also to make future changes nicer to diff 2016-09-06 Andrea Bolognani libvirt-guests.service: Improve description libvirt-guests.service does both suspend *and* resume guests, depending on whether it's being started or stopped: the description should reflect this, to avoid confusing messages during startup. Replace "active" with "running" (to match virsh list's output) and don't capitalize libvirt. 2016-09-06 Andrea Bolognani virtlogd.socket: Tie lifecycle to libvirtd.service We already guarantee that virtlogd.socket is enabled/disabled along with libvirtd.service, but if libvirtd.service has just been installed and is started before rebooting, then virtlogd.socket will not be running and guest startup will fail. Add Requires=virtlogd.socket to libvirtd.service to make sure virtlogd.socket is always started along with libvirtd.service, and add Before=libvirtd.service to both virtlogd.socket and virtlogd.service so that virtlogd never disappears before libvirtd has exited. Also add PartOf=libvirtd.service to both virtlogd.socket and virtlogd.service, so that virtlogd can be shut down when not needed. Resolves: https://bugzilla.redhat.com/1372576 2016-09-06 Daniel P. Berrange tests: add missing data files for core config 2016-09-06 Jiri Denemark qemu: Don't warn about missing device in DEVICE_DELETED event Debug priority is good enough for this. 2016-09-06 Daniel P. Berrange qemu: allow turning off QEMU guest RAM dump globally We already have the ability to turn off dumping of guest RAM via the domain XML. This is not particularly useful though, as it is under control of the management application. What is needed is a way for the sysadmin to turn off guest RAM defaults globally, regardless of whether the mgmt app provides its own way to set this in the domain XML. So this adds a 'dump_guest_core' option in /etc/libvirt/qemu.conf which defaults to false. ie guest RAM will never be included in the QEMU core dumps by default. This default is different from historical practice, but is considered to be more suitable as a default because a) guest RAM can be huge and so inflicts a DOS on the host I/O subsystem when dumping core for QEMU crashes b) guest RAM can contain alot of sensitive data belonging to the VM owner. This should not generally be copied around inside QEMU core dumps submitted to vendors for debugging c) guest RAM contents are rarely useful in diagnosing QEMU crashes 2016-09-06 Daniel P. Berrange qemu: add a max_core setting to qemu.conf for core dump size Currently the QEMU processes inherit their core dump rlimit from libvirtd, which is really suboptimal. This change allows their limit to be directly controlled from qemu.conf instead. 2016-09-06 Peter Krempa util: storage: Add json pseudo protocol support for legacy RBD strings RBD in qemu still uses only the legacy 'filename' syntax. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1371758 2016-09-06 Peter Krempa util: storage: Properly set protocol type when parsing gluster json string Commit 2ed772cd forgot to set proper protocol. This was also present in the test data. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1372251 2016-09-06 Roman Mohr docs: Add libvirt-go Go bindings to binding page 2016-09-05 Nishith Shah virsh: Complete multiple options when any one option requires data Before this patch: virsh # start --domain dom1 [TAB][TAB] <- offers filename completion virsh # start --domain [TAB][TAB] <- offers filename completion After this patch: virsh # start --domain dom1 [TAB][TAB] <- offers command completion virsh # start --domain [TAB][TAB] <- calls domain completer if defined, otherwise falls back to filename completion 2016-09-05 Nishith Shah virsh: Allow data or argument options to be completed as well virsh: Introduce usage of option completers to auto-complete arguments Call option completers if argument completion is requested using the corresponding option completer, if it is defined. 2016-09-02 Qiaowei Ren perf: add more perf events support With current perf framework, this patch adds support and documentation for more perf events, including cache misses, cache references, cpu cycles, and instructions. 2016-09-02 Qiaowei Ren perf: Adjust the perf initialization Introduce a static attr table and refactor virPerfEventEnable() for general purpose usage. This patch creates a static table/matrix that converts the VIR_PERF_EVENT_* events into their respective "attr.type" and "attr.config" so that virPerfEventEnable doesn't have the switch the calling function passes by value the 'type'. 2016-09-02 John Ferlan util: Move virPerfNew and virPerfFree Move them to the bottom under the #ifdef code. virsh: Rework the perf event names into a table. Should be easier to read virsh: Add a forward reference to perf command from domstats --perf Keep the details in one place... 2016-09-02 Qiaowei Ren util: Add some comment details for virPerfEventType Add to some details for the existing enum perf: Remove the switch from qemuDomainGetStatsPerf Remove the unnecessary switch since all VIR_PERF_EVENT* values are fetched perf: rename qemuDomainGetStatsPerfRdt() This patch rename qemuDomainGetStatsPerfRdt() to qemuDomainGetStatsPerfOneEvent() 2016-09-02 Nikolay Shirokovskiy qemu: Filter cur_balloon ABI check for certain transactions Since the domain lock is not held during preparation of an external XML config, it is possible that the value can change resulting in unexpected failures during ABI consistency checking for some save and migrate operations. This patch adds a new flag to skip the checking of the cur_balloon value and then sets the destination value to the source value to ensure subsequent checks without the skip flag will succeed. This way it is protected from forges and is keeped up to date too. 2016-09-02 Bob Liu xmconfigdata: drop tests for multi serial xen-xm doesn't support mult serial devices at all, so these tests are meaningless. xlconfigdata: add tests for multi serial Adding tests for domXML <-> xl.cfg conversions containing multiple serial devices. xenconfig: rm format/parse multi serial for xen-xm xen-xm doesn't support multi serial at all, this patch drop the domXML <-> xl.cfg conversions. 2016-09-02 Bob Liu libxl: support serial list Add support for multi serial devices, after this patch virsh can be used to connect different serial devices of running domains. E.g. vish # console --devname serial Note: This depends on a xen/libxl bug fix to have libxl_console_get_tty(...) correctly returning the tty path (as opposed to always returning the first one). [0] https://lists.xen.org/archives/html/xen-devel/2016-08/msg00438.html 2016-09-02 Jim Fehlig virpci: support driver_override sysfs interface libvirt uses the new_id PCI sysfs interface to bind a PCI stub driver to a PCI device. The new_id interface is known to be buggy and racey, hence a more deterministic interface was introduced in the 3.12 kernel: driver_override. For more details see https://www.redhat.com/archives/libvir-list/2016-June/msg02124.html For more details about the driver_override interface and examples of its usage, see https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/pci/pci-driver.c?h=v3.12&id=782a985d7af26db39e86070d28f987cad21313c0 This patch adds support for the driver_override interface by - adding new virPCIDevice{BindTo,UnbindFrom}StubWithOverride functions that use the driver_override interface - renames the existing virPCIDevice{BindTo,UnbindFrom}Stub functions to virPCIDevice{BindTo,UnbindFrom}StubWithNewid to perserve existing behavior on new_id interface - changes virPCIDevice{BindTo,UnbindFrom}Stub function to call one of the above depending on availability of driver_override The patch includes a bit of duplicate code, but allows for easily dropping the new_id code once support for older kernels is no longer desired. 2016-09-02 Cédric Bosdonnat libxl: allow vendor/product addressing for USB hostdevs libxl only has API to address the host USB devices by bus/device. Find the bus/device if the user only provided the vendor/product of the USB device. Add virHostdevFindUSBDevice to private symbols Finding an USB device from the vendor/device values will be needed by libxl driver to convert from vendor/device to bus/dev addresses. 2016-09-02 Xian Han Yu conf: Fix initialization value of 'multi' in PCI address The 'multi' element in PCI address struct used as 'virTristateSwitch', and its default value is 'VIR_TRISTATE_SWITCH_ABSENT'. Current PCI process use 'false' to initialization 'multi', which is ambiguously for assignment or comparison. This patch use '{0}' to initialize the whole PCI address struct, which fix the 'multi' initialization and makes code more simplify and explicitly. 2016-09-02 Martin Kletzander Post-release version bump to 2.3.0 2016-09-02 Daniel Veillard Release of libvirt-2.2.0 * docs/news.html.in: update for release * po/*po*: regenerate 2016-09-02 Kothapally Madhu Pavan Make --postcopy flag mandatory with --postcopy-after-precopy --postcopy-after-precopy is just an aditional flag for postcopy migration. 2016-08-31 Michal Privoznik tools: Don't list virsh-* under EXTRA_DIST When we wanted to break huge and unmaintainable virsh into smaller files first thing we did was to just move funcs into virsh-.c files and then #include them from virsh. Having it done this way we also needed to have them listed under EXTRA_DIST. However, things got changed since then and now all the virsh-*.c files are proper source files. Therefore they are listed under virsh_SOURCES too. But for some reason we forgot to remove them from EXTRA_DIST. 2016-08-29 Jim Fehlig libxl: advertise support for migration V3 The libxl driver has long supported migration V3 but has never indicated so in the connectSupportsFeature API. As a result, apps such as virt-manager that use the more generic virDomainMigrate API fail with libvirtError: this function is not supported by the connection driver: virDomainMigrate Add VIR_DRV_FEATURE_MIGRATION_V3 to the list of features marked as supported in the connectSupportsFeature API. 2016-08-29 Roman Bogorodskiy tests: fix segfault in objecteventtest Test 12 from objecteventtest (createXML add event) segaults on FreeBSD with bus error. At some point it calls testNodeDeviceDestroy() from the test driver. And it fails when it tries to unlock the device in the "out:" label of this function. Unlocking fails because the previous step was a call to virNodeDeviceObjRemove from conf/node_device_conf.c. This function removes the given device from the device list and cleans up the object, including destroying of its mutex. However, it does not nullify the pointer that was given to it. As a result, we end up in testNodeDeviceDestroy() here: out: if (obj) virNodeDeviceObjUnlock(obj); And instead of skipping this, we try to do Unlock and fail because of malformed mutex. Change virNodeDeviceObjRemove to use double pointer and set pointer to NULL. 2016-08-29 Roman Bogorodskiy bhyve: fix disks address allocation As bhyve currently doesn't use controller addressing and simply uses 1 implicit controller for 1 disk device, the scheme looks the following: pci addrees -> (implicit controller) -> disk device So in fact we identify disk devices by pci address of implicit controller and just pass it this way to bhyve in a form: -s pci_addr,ahci-(cd|hd),/path/to/disk Therefore, we cannot use virDeviceInfoPCIAddressWanted() because it does not expect that disk devices might need PCI address assignment. As a result, if a disk was specified without address, it will not be generated and domain will to start. Until proper controller addressing is implemented in the bhyve driver, force each disk to have PCI address generated if it was not specified by user. 2016-08-26 Kothapally Madhu Pavan Check for --live flag for postcopy-after-precopy migration Unlike postcopy migration there is no --live flag check for postcopy-after-precopy. 2016-08-26 Christophe Fergeau docs: Add missing / to closing tag The iothread example for virtio-scsi should be rather than for the XML to be valid. 2016-08-26 Peter Krempa conf: Fix build with picky GCC ../../src/conf/domain_conf.c:4425:21: error: potential null pointer dereference [-Werror=null-dereference] switch (vcpu->hotpluggable) { ~~~~^~~~~~~~~~~~~~ 2016-08-26 Peter Krempa qemu: driver: Validate configuration when setting maximum vcpu count Setting vcpu count when cpu topology is specified may result into an invalid configuration. Since the topology can't be modified, reject the setting if it doesn't match the requested topology. This will allow fixing the topology in case it was broken. Partially fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1370066 2016-08-26 Peter Krempa conf: Don't validate vcpu count in XML parser Validating the vcpu count is more intricate and doing it in the XML parser will make previously valid configs (with older qemus) vanish. Now that we have a very similar check in the qemu domain validation callback we can do it in a more appropriate place. This basically reverts commit b54de0830a. Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1370066 2016-08-26 Peter Krempa doc: clarify documentation for vcpu order Make it clear that vcpu order is valid for online vcpus only and state that it has to be specified for all vcpus or not provided at all. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1370043 2016-08-26 Peter Krempa qemu: driver: Fix qemuDomainHelperGetVcpus for sparse vcpu topologies ce43cca0e refactored the helper to prepare it for sparse topologies but forgot to fix the iterator used to fill the structures. This would result into a weirdly sparse populated array and possible out of bounds access and crash once sparse vcpu topologies were allowed. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1369988 2016-08-26 Peter Krempa virsh: vcpuinfo: Report vcpu number from the structure rather than it's position virVcpuInfo contains the vcpu number that the data refers to. Report what's returned by the daemon rather than the sequence number as with sparse vcpu topologies they won't match. 2016-08-26 Mikhail Feoktistov vz: getting bus type for containers We should query bus type for containers too, like for VM. In openstack we add volume disk like SCSI, so we can't hardcode SATA bus. 2016-08-26 Nikolay Shirokovskiy vz: update domain cache after device updates 2016-08-26 Olga Krishtal vz: fixed race in vzDomainAttach/DettachDevice While dettaching/attaching device in OpenStack, nova calls vzDomainDettachDevice twice, because the update of the internal configuration of the ct comes a bit latter than the update event. As the result, we suffer from the second call to dettach the same device. 2016-08-26 Pavel Glushchak vz: added VIR_MIGRATE_PARAM_BANDWIDTH param handling libvirt-python passes parameter bandwidth = 0 by default. This means that bandwidth is unlimited. VZ driver doesn't support bandwidth rate limiting, but we still need to handle it and fail if bandwidth > 0. vz: implicitly support additional migration flags * Added VIR_MIGRATE_LIVE, VIR_MIGRATE_UNDEFINE_SOURCE and VIR_MIGRATE_PERSIST_DEST to supported migration flags 2016-08-26 Laine Stump qemu: set tap device online for type='ethernet' When support for auto-creating tap devices was added to in commit 9c17d6, the code assumed that virNetDevTapCreate() would honor the VIR_NETDEV_TAP__CREATE_IFUP flag that is supported by virNetDevTapCreateInBridgePort(). That isn't the case - the latter function performs several operations, and one of them is setting the tap device online. But virNetDevTapCreate() *only* creates the tap device, and relies on the caller to do everything else, so qemuInterfaceEthernetConnect() needs to call virNetDevSetOnline() after the device is successfully created. 2016-08-26 Laine Stump qemu: remove unnecessary setting of tap device online state The linkstate setting of an is only meant to change the online status reported to the guest system by the emulated network device driver in qemu, but when support for auto-creating tap devices for was added in commit 9717d6, a chunk of code was also added to qemuDomainChangeNetLinkState() that sets the online status of the tap device (i.e. the *host* side of the interface) for type='ethernet'. This was never done for tap devices used in type='bridge' or type='network' interfaces, nor was it done in the past for tap devices created by external scripts for type='ethernet', so we shouldn't be doing it now. This patch removes the bit of code in qemuDomainChangeNetLinkState() that modifies online status of the tap device. 2016-08-26 Vasiliy Tolstov qemu: fix ethernet network type ip/route assign The call to virNetDevIPInfoAddToDev() that sets up tap device IP addresses and routes was somehow incorrectly placed in qemuInterfaceStopDevice() instead of qemuInterfaceStartDevice() in commit fe8567f6. This fixes that error by moving the call to virNetDevIPInfoAddToDev() to qemuInterfaceStartDevice(). 2016-08-24 Peter Krempa qemu: hotplug: Add support for VCPU unplug This patch removes the old vcpu unplug code completely and replaces it with the new code using device_del. The old hotplug code basically never worked with any recent qemu and thus is useless. As the new code is using device_del all the implications of using it are present. Contrary to the device deletion code, the vcpu deletion code fails if the unplug request is not executed in time. 2016-08-24 Peter Krempa qemu: hotplug: Allow marking unplugged devices by alias Add a overlay function that takes the alias directly rather than extracting it from a device info. 2016-08-24 Peter Krempa qemu: Use modern vcpu hotplug approach if possible To allow unplugging the vcpus, hotplugging of vcpus on platforms which require to plug multiple logical vcpus at once or plugging them in an arbitrary order it's necessary to use the new device_add interface for vcpu hotplug. This patch adds support for the device_add interface using the old setvcpus API by implementing an algorithm to select the appropriate entities to plug in. 2016-08-24 Peter Krempa qemu: command: Add support for sparse vcpu topologies Add support for using the new approach to hotplug vcpus using device_add during startup of qemu to allow sparse vcpu topologies. There are a few limitations imposed by qemu on the supported configuration: - vcpu0 needs to be always present and not hotpluggable - non-hotpluggable cpus need to be ordered at the beginning - order of the vcpus needs to be unique for every single hotpluggable entity Qemu also doesn't really allow to query the information necessary to start a VM with the vcpus directly on the commandline. Fortunately they can be hotplugged during startup. The new hotplug code uses the following approach: - non-hotpluggable vcpus are counted and put to the -smp option - qemu is started - qemu is queried for the necessary information - the configuration is checked - the hotpluggable vcpus are hotplugged - vcpus are started This patch adds a lot of checking code and enables the support to specify the individual vcpu element with qemu. 2016-08-24 Peter Krempa qemu: process: Copy final vcpu order information into the vcpu definition The vcpu order information is extracted only for hotpluggable entities, while vcpu definitions belonging to the same hotpluggable entity need to all share the order information. We also can't overwrite it right away in the vcpu info detection code as the order is necessary to add the hotpluggable vcpus enabled on boot in the correct order. The helper will store the order information in places where we are certain that it's necessary. 2016-08-24 Peter Krempa qemu: command: Add helper to convert vcpu definition to JSON props For use on the monitor we need to format certain parts of the vcpu private definition into a JSON object. Add a helper. qemu: migration: Prepare for non-contiguous vcpu configurations Introduce a new migration cookie flag that will be used for any configurations that are not compatible with libvirt that would not support the specific vcpu hotplug approach. This will make sure that old libvirt does not fail to reproduce the configuration correctly. 2016-08-24 Peter Krempa conf: Add XML for individual vCPU hotplug Individual vCPU hotplug requires us to track the state of any vCPU. To allow this add the following XML: ... 3 ... The 'enabled' attribute allows to control the state of the vcpu. 'hotpluggable' controls whether given vcpu can be hotplugged and 'order' allows to specify the order to add the vcpus. 2016-08-24 Peter Krempa util: Extract and rename qemuDomainDelCgroupForThread to virCgroupDelThread qemu: domain: Prepare for VCPUs vanishing while libvirt is not running Similarly to devices the guest may allow unplug of the VCPU if libvirt is down. To avoid problems, refresh the vcpu state on reconnect. Don't mess with the vcpu state otherwise. 2016-08-24 Peter Krempa qemu: domain: Extract cpu-hotplug related data Now that the monitor code gathers all the data we can extract it to relevant places either in the definition or the private data of a vcpu. As only thread id is broken for TCG guests we may extract the rest of the data and just skip assigning of the thread id. In case where qemu would allow cpu hotplug in TCG mode this will make it work eventually. 2016-08-24 Peter Krempa tests: cpu-hotplug: Add data for ppc64 without threads enabled The reported data is unusual so add it to the test suite. tests: cpu-hotplug: Add data for ppc64 out-of-order hotplug Test the algorithm that extracts the order in which the vcpu entries were plugged in on a sample of data created by plugging in vcpus arbitrarily. 2016-08-24 Peter Krempa tests: cpu-hotplug: Add data for ppc64 platform including hotplug Power 8 platform's basic hotpluggable unit is a core rather than a thread for x86_64 family. This introduces most of the complexity of the matching code and thus needs to be tested. The test data contain data captured from in-order cpu hotplug and unplug operations. 2016-08-24 Peter Krempa tests: cpu-hotplug: Add data for x86 hotplug with 11+ vcpus During review it was reported that adding at least 11 vcpus creates a collision of prefixes in the monitor matching algorithm. Add a test case to verify that the problem won't happen. tests: Add test infrastructure for qemuMonitorGetCPUInfo As the combination algorithm is rather complex and ugly it's necessary to make sure it works properly. Add test suite infrastructure for testing it along with a basic test based on x86_64 platform. 2016-08-24 Peter Krempa qemu: monitor: Add algorithm for combining query-(hotpluggable-)-cpus data For hotplug purposes it's necessary to retrieve data using query-hotpluggable-cpus while the old query-cpus API report thread IDs and order of hotplug. This patch adds code that merges the data using a rather non-trivial algorithm and fills the data to the qemuMonitorCPUInfo structure for adding to appropriate place in the domain definition. 2016-08-24 Peter Krempa qemu: monitor: Add support for calling query-hotpluggable-cpus Add support for retrieving information regarding hotpluggable cpu units supported by qemu. Data returned by the command carries information needed to figure out the granularity of hotplug, the necessary cpu type name and the topology information. Note that qemu doesn't specify any particular order of the entries thus it's necessary sort them by socket_id, core_id and thread_id to the order libvirt expects. 2016-08-24 Peter Krempa qemu: monitor: Extract QOM path from query-cpus reply To allow matching up the data returned by query-cpus to entries in the query-hotpluggable-cpus reply for CPU hotplug it's necessary to extract the QOM path as it's the only link between the two. 2016-08-24 Peter Krempa qemu: capabilities: Extract availability of new cpu hotplug for machine types QEMU reports whether 'query-hotpluggable-cpus' is supported for a given machine type. Extract and cache the information using the capability cache. When copying the capabilities for a new start of qemu, mask out the presence of QEMU_CAPS_QUERY_HOTPLUGGABLE_CPUS if the machine type doesn't support hotpluggable cpus. 2016-08-24 Peter Krempa qemu: Forbid config when topology based cpu count doesn't match the config As of qemu commit: commit a32ef3bfc12c8d0588f43f74dcc5280885bbdb30 Author: Thomas Huth Date: Wed Jul 22 15:59:50 2015 +0200 vl: Add another sanity check to smp_parse() function v2.4.0-952-ga32ef3b configuration where the maximum CPU count doesn't match the topology is rejected. Prior to that only configurations where the topology would contain more cpus than the maximum count would be rejected. Use QEMU_CAPS_QUERY_HOTPLUGGABLE_CPUS as a relevant recent enough witness to avoid breaking old configs. 2016-08-24 Peter Krempa qemu: Add capability for query-hotpluggable-cpus command qemu: monitor: Return struct from qemuMonitor(Text|Json)QueryCPUs Prepare to extract more data by returning an array of structs rather than just an array of thread ids. Additionally report fatal errors separately from qemu not being able to produce data. qemu: monitor: Return structures from qemuMonitorGetCPUInfo The function will gradually add more returned data. Return a struct for every vCPU containing the data. 2016-08-24 Chen Hanxiao storage_backend_rbd: fix typos s/failed/failed to 2016-08-24 Pino Toscano virsh: use vshError consistently after virBufferError checks If virBufferError() reports an error, then vshError() is needed to report the error situation instead of a simple vshError(). virsh: avoid i18n puzzle Use the full versions of the message, instead of composing a base message with what was updated; the change makes the messages properly translatable, since different parts of a sentence might need different declensions for example. 2016-08-24 Pino Toscano virsh: respect -q/--quiet more Turn various vshPrint() informative messages into vshPrintExtra(), so they are not printed when requesting the quiet mode; neither XML/info outputs nor the results of commands are affected. Also change the expected outputs of the virsh-undefine test, since virsh is invoked in quiet mode there. Some informative messages might still be converted (and thus silenced when in quiet mode), but this is an improvements nonetheless. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1358179 2016-08-23 Peter Krempa Fix remote_protocol-structs after recent commit Commit 0adc9d26ae0cfbb4aa227e89424f3d48bb977da0 broke the build since remote_protocol-structs was not updated. 2016-08-23 Nikolay Shirokovskiy vz: add ACL checks to API calls vzDomainMigrateConfirm3Params is whitelisted. Otherwise we need to move removing domain from domain list from perform to confirm step. This would further imply adding a flag and check that migration is in progress to prohibit mistakenly (maliciously) removing domains on confirm step. vz version of p2p also need to be fixed to include confirm step. One would also need to add means to cleanup pending migration on client disconnect as now is has state across several API calls. On the other hand current version of confirm step is totaly harmless thus it is easier to whitelist it at the moment. 2016-08-23 Nikolay Shirokovskiy remote: rename protocol names for close callbacks This way we make naming consistent to API calls and make subsequent ACL checks possible (otherwise ACL check would discover name discrepancies). vz: prepare migration for ACL checks ACL check on perform step should be in API call itself to make ACL checking script pass. Thus we need to reorganize code to obtain domain object in perform API itself. Most of this is straight forward, the only nuance is dropping locks on lengthy remote operations. The other motivation is to have only perform step ACL checks for p2p migration instead of both begin in perform if we can leave ACL check in vzDomainMigratePerformStep. vz: expand setting memory API calls We need it to prepare the calls for ACL checks otherwise ACL checking script will fail. vz: add missing flagged versions of API functions vz: factor out converting block stats to params This action deserves its own function and makes main API call structure much cleaner. vz: factor out block stats impl Now we can use intended ACL check for both API calls. vz: implement plain create API thru createFlags instead of visa versa 2016-08-23 Nikolay Shirokovskiy vz: expand start/stop/... APIs for ACL checks The original motivation is to expand API calls like start/stop etc so that the ACL checks could be added. But this patch has its own befenits. 1. functions like prlsdkStart/Stop use common routine to wait for job without domain lock. They become more self contained and do not return intermediate PRL_RESULT. 2. vzDomainManagedSave do not update cache twice. 2016-08-22 John Ferlan network: Need to free formatted addr in networkDnsmasqConfContents Commit id '0b6336c2' formatted the 'addr', but didn't VIR_FREE it. Found by Coverity. 2016-08-22 JieWang remove the dead code this patch is to remove the dead code Signed-off-by: JieWang 2016-08-20 John Ferlan qemu: Fix crash hot plugging luks volume https://bugzilla.redhat.com/show_bug.cgi?id=1367259 Crash occurs because 'secrets' is being dereferenced in call: if (qemuDomainSecretSetup(conn, priv, secinfo, disk->info.alias, VIR_SECRET_USAGE_TYPE_VOLUME, NULL, &src->encryption->secrets[0]->seclookupdef, true) < 0) (gdb) p *src->encryption $1 = {format = 2, nsecrets = 0, secrets = 0x0, encinfo = {cipher_size = 0, cipher_name = 0x0, cipher_mode = 0x0, cipher_hash = 0x0, ivgen_name = 0x0, ivgen_hash = 0x0}} (gdb) bt priv=priv@entry=0x7fffc03be160, disk=disk@entry=0x7fffb4002ae0) at qemu/qemu_domain.c:1087 disk=0x7fffb4002ae0, vm=0x7fffc03a2580, driver=0x7fffc02ca390, conn=0x7fffb00009a0) at qemu/qemu_hotplug.c:355 Upon entry to qemuDomainAttachVirtioDiskDevice, src->encryption points at a valid 'secret' buffer w/ nsecrets == 1; however, the call to qemuDomainDetermineDiskChain will call virStorageFileGetMetadata and eventually virStorageFileGetMetadataInternal where the src->encryption was overwritten when probing the volume. Commit id 'a48c7141' added code to virStorageFileGetMetadataInternal to determine if the disk/volume would use/need encryption and allocated a meta->encryption. This overwrote an existing encryption buffer already provided by the XML This patch adds a check for meta->encryption already present before just allocating and overwriting an existing buffer. It then checks the existing encryption data to ensure the XML provided format for the disk matches the expected format read from the disk and errors if there is a mismatch. 2016-08-20 Laine Stump network: allow limiting a element to certain domains For some unknown reason the original implementation of the element only took advantage of part of the functionality in the dnsmasq feature it exposes - it allowed specifying the ip address of a DNS server which *all* DNS requests would be forwarded to, like this: This is a frontend for dnsmasq's "server" option, which also allows you to specify a domain that must be matched in order for a request to be forwarded to a particular server. This patch adds support for specifying the domain. For example: would forward requests for bob.example.com, ftp.example.com and joe.corp.example.com all to the DNS server at 192.168.1.1, but would forward requests for travesty.org and www.travesty.org to 10.0.0.1. And due to the second line, requests for www.example.com, and odd.www.example.com would be resolved by the libvirt network's own DNS server (i.e. thery wouldn't be immediately forwarded) even though they also match 'example.com' - the match is given to the entry with the longest matching domain. DNS requests not matching any of the entries would be resolved by the libvirt network's own DNS server. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1331796 2016-08-20 Laine Stump network: allow disabling dnsmasq's DNS server If you define a libvirt virtual network with one or more IP addresses, it starts up an instance of dnsmasq. It's always been possible to avoid dnsmasq's dhcp server (simply don't include a element), but until now it wasn't possible to avoid having the DNS server listening; even if the network has no element, it is started using default settings. This patch adds a new attribute to : enable='yes|no'. For backward compatibility, it defaults to 'yes', but if you don't want a DNS server created for the network, you can simply add: to the network configuration, and next time the network is started there will be no dns server created (if there is dhcp configuration, dnsmasq will be started with "port=0" which disables the DNS server; if there is no dhcp configuration, dnsmasq won't be started at all). 2016-08-20 Laine Stump network: new network forward mode 'open' The new forward mode 'open' is just like mode='route', except that no firewall rules are added to assure that any traffic does or doesn't pass. It is assumed that either they aren't necessary, or they will be setup outside the scope of libvirt. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=846810 2016-08-19 Michal Privoznik networkxml2conftest: Don't leak dnsmasq capabilities ==18324== 32 bytes in 1 blocks are still reachable in loss record 41 of 114 ==18324== at 0x4C2C070: calloc (vg_replace_malloc.c:623) ==18324== by 0x4EA479B: virAlloc (viralloc.c:144) ==18324== by 0x4EA674A: virBitmapNewQuiet (virbitmap.c:77) ==18324== by 0x4EA67F7: virBitmapNew (virbitmap.c:106) ==18324== by 0x4EC777D: dnsmasqCapsNewEmpty (virdnsmasq.c:801) ==18324== by 0x4EC781B: dnsmasqCapsNewFromBuffer (virdnsmasq.c:815) ==18324== by 0x407CF4: mymain (networkxml2conftest.c:99) ==18324== by 0x409CF0: virTestMain (testutils.c:982) ==18324== by 0x4080EA: main (networkxml2conftest.c:136) 2016-08-18 Mikhail Feoktistov vz: add validation callbacks This patch fixes a bug which occurs when we check a bus and unit number for a new attached disk. We should do this check in ValidadionCallback, not in PostParse callback. Because in PostParse we have not initialized disk->info.addr.drive struct yet. Move part of code from domainPostParseCallback to domainValidateCallback and part from devicesPostParseCallback to deviceValidateCallback. PostParse callbacks are for modification data. ValidateCallbacks are only for checks. 2016-08-18 Olga Krishtal vz: fixed race in vzDomainAttach/DettachDevice While dettaching/attaching device in OpenStack, nova calls vzDomainDettachDevice twice, because the update of the internal configuration of the ct comes a bit latter than the update event. As the result, we suffer from the second call to dettach the same device. 2016-08-18 Maxim Nestratov vz: specify VIR_DOMAIN_NET_TYPE_NETWORK for routed networks Somehow we lost this during recent refactoring vz: reset errors after ignoring return values If we are going to ignore return value of a functions that can raise an error, it's not enough to use ignore_value construction. We should explicitly call virResetLastError vz: don't fail query domain info in case we don't have valid stats handle vz: get additional error information from job correctly First, make function logPrlEventErrorHelper be void and only print information (if any) from an event. Second, don't rewrite original error with any errors we get during parsing event info. Third, ignore PRL_ERR_NO_DATA at all. 2016-08-18 Ján Tomko cfg.mk: join not_streq and not_strneq tests The marginally nicer error message is not worth the extra lines in cfg.mk. Also drop the excludes since there was only one offender in the tests. 2016-08-18 Ján Tomko cfg.mk: drop redundant sc_prohibit_gethostby Both gethostbyaddr and gethostbyname* are already checked by sc_prohibit_nonreentrant. cfg.mk: use subst instead of tr GNU make is able to replace characters, no need to call tr. tests: fix the return value of test-wrap-argv The script was returning success unless it failed on the last file. This went unnoticed because sc_prohibit_long_lines forbids lines longer than 90 characters in .arg[sv] files. maint: update to latest gnulib Pick up the new syntax-check speedups. 2016-08-17 Ján Tomko Introduce QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY Check whether the disable-legacy property is present on the following devices: virtio-balloon-pci virtio-blk-pci virtio-scsi-pci virtio-serial-pci virtio-9p-pci virtio-net-pci virtio-rng-pci virtio-gpu-pci virtio-input-host-pci virtio-keyboard-pci virtio-mouse-pci virtio-tablet-pci Assuming that if QEMU knows other virtio devices where this property is applicable, it will have at least one of these devices. Added in QEMU by: commit e266d421490e0ae83044bbebb209b2d3650c0ba6 virtio-pci: add flags to enable/disable legacy/modern 2016-08-17 John Ferlan qemu: Fix the command line generation for rbd auth using aes secrets https://bugzilla.redhat.com/show_bug.cgi?id=1182074 Since libvirt still uses a legacy qemu arg format to add a disk, the manner in which the 'password-secret' argument is passed to qemu needs to change to prepend a 'file.' If in the future, usage of the more modern disk format, then the prepended 'file.' can be removed. Fix based on Jim Fehlig posting and subsequent upstream list followups, see: http://www.redhat.com/archives/libvir-list/2016-August/msg00777.html for details. Introduced by commit id 'a1344f70'. 2016-08-17 Chen Hanxiao qemu_driver: update comments for qemuDomainSaveInternal s/libvirt.c/libvirt-domain.c 2016-08-17 John Ferlan conf: Provide error on undefined vcpusched entry Modify virDomainDefGetVcpuSched to emit an error message if virDomainDefGetVcpu returns NULL meaning the vcpu could not be found. Prior to commit id '9cc931f0b' the error message would have been issued in virDomainDefGetVcpu. 2016-08-17 Pavel Hrdina qemu_process: graphics: setup listen types before ports are reserved/allocated The code that setups listen types may change a listen type from address to socket based on configuration from qemu.conf. This needs to be done before we reserve/allocate ports that won't be used. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1364843 2016-08-17 Pavel Hrdina qemu_process: graphics: reserve port only if listen type is address or network Ports are valid only for listen types 'address' and 'network', other listen types doesn't use them so we should not try to reserve any ports. qemu_process: graphics: extract for loop out of qemuProcessGraphicsReservePorts qemu_process: graphics: extract port allocation into function qemu_process: graphics: ref driver config only in function where it is used Signed-off-by: Pavel Hrdina util: Make virStringArrayHasString() const-correct The first argument should be const char ** instead of char **, because this is a search function and as such it doesn't, and shouldn't, alter the haystack in any way. This change means we no longer have to cast arrays of immutable strings to arrays of mutable strings; we still have to do the opposite, though, but that's reasonable. 2016-08-16 John Ferlan conf: Provide error on undefined iothreadsched entry When commit id '6dfb4507' refactored where the iothreadsched data was stored, the error message for when the virDomainIOThreadIDFind failed to find an iothreadid ("iothreadsched attribute 'iothreads' uses undefined iothread ids") was lost. This led to the possibility that someone would try to use it, but receive the generic message "An error occurred, but the cause is unknown". This patch adds the error message back so that someone will know that they have an invalid configuration. 2016-08-16 Peter Krempa qemu: driver: Split out regular vcpu hotplug code into a function All other modes of qemuDomainSetVcpusFlags have helpers so finish the work by splitting the regular code into a new function. This patch also touches up the coding (spacing) style. 2016-08-16 Peter Krempa qemu: driver: Extract setting of live vcpu count The live code does ugly things. Contain it in a separate function. qemu: setvcpus: Extract setting of maximum vcpu count Setting of the maximum vcpu count is slightly semantically different thus split it into a self-contained func. 2016-08-16 Peter Krempa utils: storage: Fix JSON field name for uri based storage qemu uses 'url' instead of 'uri'. They unfortunately look very similar. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1367260 2016-08-16 Ján Tomko conf: report an error message for non-existing USB hubs If any of the devices referenced a USB hub that does not exist, defining the domain would either fail with: error: An error occurred, but the cause is unknown (if only the last hub in the path is missing) or crash. Return a proper error instead of crashing. https://bugzilla.redhat.com/show_bug.cgi?id=1367130 2016-08-16 Ján Tomko conf: free the ports array of a USB hub The array needs to be freed too, not just its members. https://bugzilla.redhat.com/show_bug.cgi?id=1366097 2016-08-16 Peter Krempa qemu: setcpus: Report better errors Mention whether it was the live or persistent definition which caused an error reported and explicitly error out in case when attempting to set maximum vcpu count for a live domain. 2016-08-16 Daniel P. Berrange lxc: don't try to reference NULL when mounting filesystems would lead to lxcContainerMountAllFS calling STRPREFIX on a NLL pointer because it failed to check if fs->src->path was non-NULL. This is a regression caused by commit da665fbd4858890fbb3bbf5da2a7b6ca37bb3220 Author: Olga Krishtal Date: Thu Jul 14 16:52:38 2016 +0300 filesystem: adds possibility to use storage pool as fs source Signed-off-by: Olga Krishtal 2016-08-16 Daniel P. Berrange lxc: don't try to resolve a NULL path for filesystems would lead to lxcContainerResolveSymlinks calling access(NULL) because it failed to check if fs->src->path was non-NULL. This is a regression caused by commit da665fbd4858890fbb3bbf5da2a7b6ca37bb3220 Author: Olga Krishtal Date: Thu Jul 14 16:52:38 2016 +0300 filesystem: adds possibility to use storage pool as fs source Signed-off-by: Olga Krishtal 2016-08-16 Michal Privoznik libxl_driver: Indent LIBXL_VBD_SECTOR_SIZE macro correctly Because of change in caaa1bd357a99ad this macro is no under #ifdef block. That means it needs to be re-intended correctly. 2016-08-16 Roman Bogorodskiy tests: fix domaincapstest linking for libxl Commit 11567cf added some libxl tests into domaincapstest and added libvirt_driver_libxl_impl.la to domaincapstest_LDADD. This causes link fail on systems without GNU regex implementation: gmake[2]: Entering directory '/usr/home/novel/code/libvirt/tests' CCLD domaincapstest ../src/.libs/libvirt_driver_libxl_impl.a(libvirt_driver_libxl_impl_la-libxl_capabilities.o): In function `libxlMakeCapabilities': libxl/libxl_capabilities.c:(.text+0x6b2): undefined reference to `rpl_regcomp' libxl/libxl_capabilities.c:(.text+0x6d0): undefined reference to `rpl_regerror' libxl/libxl_capabilities.c:(.text+0x803): undefined reference to `rpl_regexec' libxl/libxl_capabilities.c:(.text+0xa58): undefined reference to `rpl_regfree' clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation) This happens because on these system it tries to use gnulib's builtin regex implementation, but doesn't link to gnulib. Fix by adding $(GNULIB_LIBS) along with libvirt_driver_libxl_impl.la to domaincapstest_LDADD. 2016-08-16 Roman Bogorodskiy libxl: fix unused functions Commit eee7bd4e introduced two functions: libxlDiskPathToID and libxlDiskSectorSize. However, as they're used only by code under #ifdef __linux__, on non-Linux platforms it results in errors similar to this: CC libxl/libvirt_driver_libxl_impl_la-libxl_driver.lo libxl/libxl_driver.c:5263:1: error: unused function 'libxlDiskPathToID' [-Werror,-Wunused-function] libxlDiskPathToID(const char *virtpath) ^ libxl/libxl_driver.c:5312:1: error: unused function 'libxlDiskSectorSize' [-Werror,-Wunused-function] libxlDiskSectorSize(int domid, int devno) ^ 2 errors generated. Fix that by moving these functions under the #ifdef __linux__ block. 2016-08-15 Jovanka Gulicoska Introduce node device update event as top level event This event is emitted when a nodedev XML definition is updated, like when cdrom media is changed in a cdrom block device. Also includes node device update event implementation for udev backend, virsh nodedev-event support, and event-test support 2016-08-15 John Ferlan virsh: Fix core for cmdSecretGetValue https://bugzilla.redhat.com/show_bug.cgi?id=1366611 When commit id 'cb2e3e50' reworked the cmdSecretGetValue call to use VIR_DISPOSE_STRING for base64, it neglected to initialize the base64 value to NULL since the cleanup: label could be reached prior to the base64 value being set or not. This resulted in a core dump, adding the initialization will avoid the issue. 2016-08-15 Pavel Hrdina qemu_command: don't modify heads for graphics device Setting heads to 0 in case that *max_outputs* is not supported while building command line doesn't have any real effect. It only removes *heads* attribute from live XML, but after restarting libvirt the default value is restored. 2016-08-15 Michal Privoznik virschematest: Make sure that validator is initialized It may happen that a developer wants to run just a specific subset of tests: tests $ VIR_TEST_RANGE=22 ../run ./virschematest This now fails miserably: ==6840== Invalid read of size 8 ==6840== at 0x4F397C0: virXMLValidatorValidate (virxml.c:1216) ==6840== by 0x402B72: testSchemaFile (virschematest.c:53) ==6840== by 0x403737: virTestRun (testutils.c:180) ==6840== by 0x402CF5: testSchemaDir (virschematest.c:98) ==6840== by 0x402EB1: testSchemaDirs (virschematest.c:131) ==6840== by 0x40314D: mymain (virschematest.c:194) ==6840== by 0x4051AF: virTestMain (testutils.c:982) ==6840== by 0x4035A9: main (virschematest.c:217) ==6840== Address 0x10 is not stack'd, malloc'd or (recently) free'd Problem is, we are trying to do two types of tests here: validate RNG schema itself, and validate XML files against RNG schemas. And the latter tries to re-use a resource allocated in the former. Therefore if the former is skipped (due to VIR_TEST_RANGE) we have to allocate the resource manually. 2016-08-13 Jim Fehlig cpu_x86: fix libvirtd crash when host cpu vendor is not available When starting a guest and copying host vendor cpuid to the guest cpu, libvirtd would crash if the host cpu contained a NULL vendor field. Avoid the crash by checking for a valid vendor in the host cpu before copying the cpuid to the guest cpu. For completeness, here is a backtrace from the crash (gdb) bt f0 0x00007ffff739bf33 in x86DataCpuid (cpuid=0x8, cpuid=0x8, data=data@entry=0x7fffb800ee78) at cpu/cpu_x86.c:287 f1 virCPUx86DataAddCPUID (data=data@entry=0x7fffb800ee78, cpuid=0x8) at cpu/cpu_x86.c:355 f2 0x00007ffff739ef47 in x86Compute (host=, cpu=0x7fffb8000cc0, guest=0x7fffecca7348, message=) at cpu/cpu_x86.c:1580 f3 0x00007fffd2b38e53 in qemuBuildCpuModelArgStr (migrating=false, hasHwVirt=, qemuCaps=0x7fffb8001040, buf=0x7fffecca7360, def=0x7fffc400ce20, driver=0x1c) at qemu/qemu_command.c:6283 f4 qemuBuildCpuCommandLine (cmd=cmd@entry=0x7fffb8002f60, driver=driver@entry=0x7fffc80882c0, def=def@entry=0x7fffc400ce20, qemuCaps=qemuCaps@entry=0x7fffb8001040, migrating=) at qemu/qemu_command.c:6445 (gdb) f2 (gdb) p *host_model $23 = {name = 0x7fffb800ec50 "qemu64", vendor = 0x0, signature = 0, data = { len = 2, data = 0x7fffb800e720}} 2016-08-12 Andrea Bolognani qemu: command: Simplify USB controller model selection Since we now pick the default USB controller model when parsing the guest XML, we can get rid of some duplicated code so that the default model selection happens in one place only. Add some comments as well. 2016-08-12 Andrea Bolognani qemu: domain: Drop piix3-ohci controller for migration Now that the default USB controller model is explicit rather than implicit for i440fx machines, we have to tweak the conditions for dropping it in order to keep migration towards libvirt <= 0.9.4 working. 2016-08-12 Andrea Bolognani qemu: domain: Reflect USB controller model in guest XML When the user doesn't specify any model for a USB controller, we use an architecture-dependent default, but we don't reflect it in the guest XML. Pick the default USB controller model when parsing the guest XML instead of when creating the QEMU command line, so that our choice is saved back to disk. 2016-08-12 Michal Privoznik virschematest: Initialize @data ==8630== Invalid read of size 8 ==8630== at 0x4EA4F0F: virFree (viralloc.c:582) ==8630== by 0x4F398F0: virXMLValidatorFree (virxml.c:1257) ==8630== by 0x40305C: mymain (virschematest.c:191) ==8630== by 0x405159: virTestMain (testutils.c:982) ==8630== by 0x403553: main (virschematest.c:215) ==8630== Address 0xcd72243 is 131 bytes inside a block of size 177 free'd ==8630== at 0x4C2B1F0: free (vg_replace_malloc.c:473) ==8630== by 0x4EA4F19: virFree (viralloc.c:582) ==8630== by 0x4ED0973: virFindFileInPath (virfile.c:1646) ==8630== by 0x405149: virTestMain (testutils.c:980) ==8630== by 0x403553: main (virschematest.c:215) 2016-08-12 Michal Privoznik schema: Don't validate paths https://bugzilla.redhat.com/show_bug.cgi?id=1353296 On UNIX like systems there are no constraints on what characters can be in file/dir names (except for NULL, obviously). Moreover, some values that we think of as paths (e.g. disk source) are not necessarily paths at all. For instance, some hypervisors take that as an arbitrary identifier and corresponding file is then looked up by hypervisor in its table. Instead of trying to fix our regular expressions (and forgetting to include yet another character there), lets drop the validation completely. 2016-08-10 Michal Privoznik virNetDevMacVLanCreateWithVPortProfile: Drop @ret Usually, this variable is used to hold the return value for a function of ours. Well, this is not the case. Its use does not match our pattern and therefore it is very misleading. Drop it and define an alternative @rc variable, but only in that single block where it is needed. virNetDevMacVLanCreateWithVPortProfile: Drop @rc This variable is very misleading. We use VIR_FORCE_CLOSE to set it to -1 and returning it even though it does not refer to a FD at all. It merely holds 0 or -1. Drop it completely. Also, at the same time some corner cases are fixed too. 2016-08-10 Michal Privoznik virNetDevMacVLanCreateWithVPortProfile: Don't mask virNetDevMacVLanTapOpen error https://bugzilla.redhat.com/show_bug.cgi?id=1240439 In this function we create a macvtap device and open its tap device. Possibly multiple times. Now the thing is, if opening the tap device fails, that is virNetDevMacVLanTapOpen() returns a negative value, we unroll all the changes BUT return 0 fooling caller into thinking everything went okay. 2016-08-10 Cole Robinson qemu: fix qemu.conf security_driver Since a9331394 (first release v2.1.0), specifying a manual security_driver setting in qemu.conf causes the daemon to fail to start, erroring with 'Duplicate security driver X'. The duplicate checking was incorrectly comparing every entry against itself, guaranteeing a false positive. https://bugzilla.redhat.com/show_bug.cgi?id=1365607 2016-08-10 Laine Stump conf: restrict expander buses to connect only to a root bus More misunderstanding/mistaken assumptions on my part - I had thought that a pci-expander-bus could be plugged into any legacy PCI slot, and that pcie-expander-bus could be plugged into any PCIe slot. This isn't correct - they can both be plugged ontly into their respective root buses. This patch adds that restriction. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1358712 2016-08-10 Laine Stump conf: restrict where dmi-to-pci-bridge can be connected libvirt had allowed a dmi-to-pci-bridge to be plugged in anywhere a normal PCIe endpoint can be connected, but this is wrong - it will only work if it's plugged into pcie-root (the PCIe root complex) or a pcie-expander-bus (the qemu device pxb-pcie). This patch adjusts the connection flags accordingly. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1363648 2016-08-10 Laine Stump conf: don't allow connecting upstream-port directly to pce-expander-bus I apparently misunderstood Marcel's description of what could and couldn't be plugged into qemu's pxb-pcie controller (known as pcie-expander-bus in libvirt) - I specifically allowed directly connecting a pcie-switch-upstream-port, and it turns out that causes the guest kernel to crash. This patch forbids such a connection, and updates the xml docs appropriately. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1361172 2016-08-10 Laine Stump conf: improve error log when PCI devices don't match requested controller The virDomainPCIAddressFlagsCompatible() error logs report that a device required a controller that accepted standard PCI endpoint devices, or PCI Express endpoint devices, and if hotplug was required by the configuration but not provided by the selected controller. But the wording of the error messages was apparently confusing (according to the bugzilla report referenced below). On top of that, if the device was something other than an endpoint device (e.g. a pcie-switch-downstream-port) the error message was a complete punt - it would just say that the flags were incorrect. This patch makes the messages for PCI/PCIe endpoint and hotplug requirements more clear, and also specifically indicates what was the device type when it is other than an endpoint device. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1363627 2016-08-10 Erik Skultety virt-admin: Fix the error when an invalid URI has been provided After commit 9d479dd1 fiddled with the cmdConnect's output which used to be a bit more verbose prior to the mentioned commit, the program flow would result in a quite confusing error if an invalid URI has been provided: error: Failed to connect to the admin server Connected to the admin server error: The problem is that the commit mentioned above relied on the fact that connect routine always succeeds which is not true. 2016-08-10 Jiri Denemark cpu_x86: Fix host-model CPUs on hosts with CMT Since the introduction of CMT features (commit v1.3.5-461-gf294b83) starting a domain with host-model CPU on a host which supports CMT fails because QEMU complains about unknown 'cmt' feature: qemu-system-x86_64: CPU feature cmt not found https://bugzilla.redhat.com/show_bug.cgi?id=1355857 2016-08-10 Jiri Denemark tests: Add a test for host-model CPU with CMT feature The generated command line wouldn't work since QEMU doesn't know what 'cmt' is. The following patch will fix this issue. https://bugzilla.redhat.com/show_bug.cgi?id=1355857 2016-08-10 Jiri Denemark cpu_x86: Properly drop non-migratable features By removing a non-migratable feature in a for loop we would fail to drop every second non-migratable feature if the features array contained several of them in a row. cpu_x86: Introduce x86FeatureIsMigratable 2016-08-10 Chen Hanxiao virsh: clarify snapshot --live In libvirt, snapshot means disk snapshot. snapshot --live is more like VM checkpoint. Make it clear in virsh.pod. 2016-08-09 Jovanka Gulicoska virsh: Introduce nodedev-event command Add nodedev-event support for node device lifecycle events 2016-08-09 Erik Skultety virt-admin: Properly fix the default session daemon URI to admin server Commit 30ce2f0e tried to fix the issue with an incorrect session URI to admin server but it messed up the checks: if (geteuid == 0 && VIR_STRDUP(*uristr, "libvirtd:///system") < 0) return -1; else if (VIR_STRDUP(*uristr, "libvirtd:///session") < 0) return -1; So if a client executed with root privileges tries to connect, its euid is checked (true) and the correct URI is successfully copied to @uristr (false), therefore the 'else' branch is taken and @uristr is replaced by the session URI which for root results in: Failed to connect socket to '/root/.cache/libvirt/libvirt-admin-sock': No such file or directory 2016-08-09 Erik Skultety admin: Fix the default uri for session daemon to libvirtd:///session Just like we decide on which URI we go with based on EUID for qemu in remote driver, do a similar thing for admin except we do not spawn a daemon in this case. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1356858 2016-08-08 Michal Privoznik docs: Distribute subsite.xsl So, I've ran into very interesting problem lately. When doing the following, I've encountered an error: libvirt.git $ make dist && tar -xJf libvirt-2.2.0.tar.xz && \ cd libvirt-2.2.0 && ./configure && \ rm docs/formatdomain.html && make -C docs make: Entering directory 'docs' make: *** No rule to make target 'formatdomain.html', needed by 'web'. Stop. make: Leaving directory 'docs' I had no idea what was going on, so I've nailed down the commit that "broke it" via running git-bisect. It was this one: 7659bd9221b9dd1cdf. But that shed no more light until I realized that the commit might actually just exposed a problem we had. And guess what - I've nailed it down. Of course we are not distributing subsite.xsl that's why make prints error message. Very misleading one I must say. 2016-08-08 Marc Hartmayer virsystemd: Fix error check Commit b3e4401dc620 introduced a check to ignore an error if the guest is already terminated. However the check accidentally compared error.code with VIR_ERR_ERROR, which is an error level, not an error code. Because of this, almost every error got silently ignored. Fixes: b3e4401dc620 ("systemd: don't report an error if the guest is already terminated") Reviewed-by: Sascha Silbe Reviewed-by: Boris Fiuczynski 2016-08-08 Kai Kang nsslinktest: also build virAtomic.h When build for architecture that don't use gcc atomic ops but pthread, it fails to build for armel: | ../tools/nss/.libs/libnss_libvirt_impl.a(libvirt_nss_la-virobject.o): In function `virClassNew': | /buildarea2/kkang/builds/qemuarm-Aug03/bitbake_build/tmp/work/armv5e-wrs-linux-gnueabi/libvirt/1.3.5-r0/build/src/../../libvirt-1.3.5/src/util/virobject.c:153: undefined reference to `virAtomicLock' | ../tools/nss/.libs/libnss_libvirt_impl.a(libvirt_nss_la-virobject.o): In function `virObjectNew': | /buildarea2/kkang/builds/qemuarm-Aug03/bitbake_build/tmp/work/armv5e-wrs-linux-gnueabi/libvirt/1.3.5-r0/build/src/../../libvirt-1.3.5/src/util/virobject.c:205: undefined reference to `virAtomicLock' | ../tools/nss/.libs/libnss_libvirt_impl.a(libvirt_nss_la-virobject.o): In function `virObjectUnref': | /buildarea2/kkang/builds/qemuarm-Aug03/bitbake_build/tmp/work/armv5e-wrs-linux-gnueabi/libvirt/1.3.5-r0/build/src/../../libvirt-1.3.5/src/util/virobject.c:277: undefined reference to `virAtomicLock' | ../tools/nss/.libs/libnss_libvirt_impl.a(libvirt_nss_la-virobject.o): In function `virObjectRef': | /buildarea2/kkang/builds/qemuarm-Aug03/bitbake_build/tmp/work/armv5e-wrs-linux-gnueabi/libvirt/1.3.5-r0/build/src/../../libvirt-1.3.5/src/util/virobject.c:298: undefined reference to `virAtomicLock' | collect2: error: ld returned 1 exit status It is similar with: http://libvirt.org/git/?p=libvirt.git;a=commit;h=12dc729 2016-08-06 Nikolay Shirokovskiy vz: add vzDomainGetJobStats vz: add getting job info for migration Unfortunately vz sdk do not provide detail information on migration progress, only progress percentage. Thus vz driver provides percents instead of bytes in data fields of virDomainJobInfoPtr. 2016-08-06 Eric Blake build: centralize use of extra Cygwin LDFLAGS The build was failing with: CCLD lockd.la libtool: error: can't build i686-pc-cygwin shared library unless -no-undefined is specified Rather than add yet another $(CYGWIN_EXTRA_LDFLAGS) to all the impacted *_la_LDFLAGS, it was easier to just pull the extra flags into ALL libraries via AM_LDFLAGS. Then, fix lockd_la_LDFLAGS to include AM_LDFLAGS, like all other libraries. 2016-08-06 Eric Blake build: use XDR_CFLAGS for Cygwin build Without XDR_CFLAGS, compilation on Cygwin fails with: CC libvirt_driver_la-libvirt-stream.lo In file included from libvirt-stream.c:26:0: rpc/virnetprotocol.h:9:21: fatal error: rpc/rpc.h: No such file or directory 2016-08-05 Michal Privoznik lxcDomainCreateXMLWithFiles: Avoid crash https://bugzilla.redhat.com/show_bug.cgi?id=1363773 Imagine that you're creating a transient domain, but for some reason, starting it fails. That is virLXCProcessStart() returns an error. With current code, in the error handling code the domain object is removed from the domain object list, @vm is set to NULL and controls jump to enjob label where virLXCDomainObjEndJob() is called which dereference vm leading to instant crash. The fix is to end the job in the error handling code and only after that remove the domain from the list and jump onto cleanup label instead of endjob. 2016-08-05 John Ferlan storage: Don't remove the pool for buildPool failure in storagePoolCreate https://bugzilla.redhat.com/show_bug.cgi?id=1362349 When adding the ability to build the pool during the start pool processing using the similar flags as buildPool processing would use, the code was essentially cut-n-pasted from storagePoolCreateXML. However, that included a call to virStoragePoolObjRemove which shouldn't happen within the storagePoolCreate path since that'll remove the pool from the list of pools only to be rediscovered if libvirtd restarts. So on failure, just fail and return as we should expect 2016-08-05 Ján Tomko Fix indentation 2016-08-05 Jiri Denemark qemu: Add tests for virQEMUCapsNewCopy Doing a load, copy, format cycle on all QEMU capabilities XML files should make sure we don't forget to update virQEMUCapsNewCopy when adding new elements to QEMU capabilities. qemu: Copy missing QEMU caps elements qemu: Copy QEMU caps elements in order of definition 2016-08-05 Peter Krempa tests: qemucapabilities: Add data for qemu 2.7.0 As of (v2.7.0-rc1-52-g42e0d60) 2016-08-05 Erik Skultety storage: Fix a NULL ptr dereference in virStorageBackendCreateQemuImg There was a missing check for vol->target.encryption being NULL at one particular place (modified by commit a48c71411) which caused a crash when user attempted to create a raw volume using a non-raw file volume as source. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1363636 2016-08-05 Peter Krempa internal: Introduce macro for stealing pointers VIR_STEAL_PTR copies the pointer from the second argument into the first argument and then sets the second to NULL. 2016-08-04 John Ferlan tests: Fix broken build Commit id 'f522b7d2' caused a build failure : GEN check-augeas-virtlogd Test failure:test_libvirtd_qemu.aug:69.3-147.28: Expected: { ... { "nvram" { "1" = "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd" } { "2" = "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd" } } ... Actual: ... { { "nvram" { "1" = "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd" } { "2" = "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd" } { "3" = "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd" } } ... This patch adds the OVMF_CODE.secboot.fd to the aug.in file 2016-08-04 Boris Fiuczynski qemu: fix capability counting 2016-08-04 Michal Privoznik qemu: Advertise OVMF_CODE.secboot.fd 2016-08-04 Michal Privoznik qemu: Enable secure boot In qemu, enabling this feature boils down to adding the following onto the command line: -global driver=cfi.pflash01,property=secure,value=on However, there are some constraints resulting from the implementation. For instance, System Management Mode (SMM) is required to be enabled, the machine type must be q35-2.4 or later, and the guest should be x86_64. While technically it is possible to have 32 bit guests with secure boot, some non-trivial CPU flags tuning is required (for instance lm and nx flags must be prohibited). Given complexity of our CPU driver, this is not trivial. Therefore I've chosen to forbid 32 bit guests for now. If there's ever need, we can refine the check later. 2016-08-04 Michal Privoznik Introduce @secure attribute to os loader element This element will control secure boot implemented by some firmwares. If the firmware used in does support the feature we must tell it to the underlying hypervisor. However, we can't know whether loader does support it or not just by looking at the file. Therefore we have to have an attribute to the element where users can tell us whether the firmware is secure boot enabled or not. Introduce SMM feature Since its release of 2.4.0 qemu is able to enable System Management Module in the firmware, or disable it. We should expose this capability in the XML. Unfortunately, there's no good way to determine whether the binary we are talking to supports it. I mean, if qemu's run with real machine type, the smm attribute can be seen in 'qom-list /machine' output. But it's not there when qemu's run with -M none. Therefore we're stuck with version based check. qemuBuildMachineCommandLine: Follow our pattern We use 'goto cleanup' for a reason. If a function can exit at many places but doesn't follow the pattern, it has to copy the free code in multiple places. 2016-08-04 Daniel P. Berrange rpm: set TLS priority to @LIBVIRT,SYSTEM on Fedora >= 25 With newest gnutls available in Fedora 25/rawhide, it is possible to have TLS priority fallbacks, so we can finally use --tls-priority=@LIBVIRT,SYSTEM 2016-08-04 Andrea Bolognani tests: qemuxml2xml: Add some USB test cases All these configurations are already covered for qemuxml2argv, but there were no equivalent tests for qemuxml2xml. tests: qemuxml2xml: Use DO_TEST() for most tests Now that DO_TEST() can be passed capabilities, there is little need to use DO_TEST_FULL() instead of DO_TEST(). tests: qemuxml2xml: Pass capabilities to DO_TEST() This will allow us to remove most DO_TEST_FULL() usages. For the time being, just add the extra argument to all DO_TEST() calls. tests: qemuxml2xml: Use WHEN_BOTH for most tests A bunch of cases were only being tested for WHEN_ACTIVE or WHEN_INACTIVE. Use WHEN_BOTH for all except the very few that actually require the existing setup. tests: qemuxml2xml: Clean up disk-mirror Instead of testing it twice using WHEN_ACTIVE and WHEN_INACTIVE separately, just use WHEN_BOTH. tests: qemuxml2argv: Fix usb-too-long-port-path-invalid The test case uses DO_TEST_PARSE_FLAGS_ERROR(), but doesn't pass any parse flag. Use DO_TEST_PARSE_ERROR() instead. tests: qemuxml2argv: Remove useless GIC flags DO_TEST_FAILURE() doesn't take a GIC version, but the GIC flag was passed anyway. Get rid of all such occurrences. 2016-08-04 Michal Privoznik virObjectEventNew: Use virObjectUnref() to free virObjectEvent While no leak was observed yet, there might be one if virObjectEventClass is ever derived from another class. Because in that case plain VIR_FREE() will not call dispose() from parent classes possibly leaking some memory. libxlDoMigrateReceive: Drop useless check for !vm In the cleanup path, @vm cannot be possibly NULL. If it were so, we would receive SIGSEGV much earlier. At the beginning of the function we do libxlDomainObjBeginJob(.., vm, ..); and so on. virshConnect: Don't leak polkit agent In our attempts to reconnect, we may create a polkit daemon. However, it may happen that we would rewrite the variable that already holds pointer to the agent. securityselinuxlabeltest: Prefer virGetLastErrorMessage() over virGetLastError At the beginning of the test, some preparation work is done. For instance new virSecurityManager is created. If this fails for whatever reason, we try to fetch the latest error and print the error message contained in it. However, if there's a bug in our code and no error is reported, this approach will lead to crash, while with virGetLastErrorMessage() it won't. virqemu: Reflect return type of virJSONValueArraySize() The virJSONValueArraySize() function return ssize_t (with possibly returning -1 if the passed json is not an array). Storing the return value into size_t is possibly dangerous then. 2016-08-04 Peter Krempa qemu: domain: Simplify return values of qemuDomainRefreshVcpuInfo Call the vcpu thread info validation separately to decrease complexity of returned values by qemuDomainRefreshVcpuInfo. This function now returns 0 on success and -1 on error. Certain failures of qemu to report data are still considered as success. Any error reported now is fatal. 2016-08-04 Peter Krempa qemu: domain: Improve vCPU data checking in qemuDomainRefreshVcpu Validate the presence of the thread id according to state of the vCPU rather than just checking the vCPU count. Additionally put the new validation code into a separate function so that the information retrieval can be split from the validation. qemu: monitor: Rename qemuMonitor(JSON|Text)GetCPUInfo Use a name that contains the command used to get the information. qemu: domain: Rename qemuDomainDetectVcpuPids to qemuDomainRefreshVcpuInfo The function will eventually do more useful stuff than just detection of thread ids. qemu: Improve error message in virDomainGetVcpus If the VM is offline we can't retrieve the runtime statistical information. Pinning could be retrieved but there are separate APIs for that. qemu: monitor: Add do-while block to QEMU_CHECK_MONITOR_FULL Assure that it's just one statement to avoid problems when used with conditions. qemu: monitor: Add monitor API for device_add supporting JSON objects Rather than formatting a string and splitting it back to a JSON object add API that will take a JSON object directly. 2016-08-04 Cole Robinson conf: events: Fix coverity warning Since 2bfa75134 virObjectEventNew can be passed a NULL 'uuid' value, so drop the ATTRIBUTE_NONNULL annotation 2016-08-03 Jim Fehlig libxl: allow libxl to calculate shadow mem requirements Long, long ago before libxl_get_required_shadow_memory() was made publicly available, its code was copied to the libxl driver for calculating shadow memory requirements of HVM domains. Long ago, libxl_get_required_shadow_memory() was exported in libxl_utils.h and included in xen-devel packages everywhere. Remove the copied code, which has become stale, and let libxl provode a proper shadow memory value. 2016-08-03 John Ferlan qemu: Add support to get/set IOThread period and quota cgroup values https://bugzilla.redhat.com/show_bug.cgi?id=1356937 Add support for IOThread quota/bandwidth and period parameters for non session mode. If in session mode, then error out. Uses all the same places where {vcpu|emulator|global}_{period|quota} are adjusted and adds the iothread values. 2016-08-03 John Ferlan conf: Add IOThread quota and period scheduler/cputune defs https://bugzilla.redhat.com/show_bug.cgi?id=1356937 Add the definitions to allow for viewing/setting cgroup period and quota limits for IOThreads. This is similar to the work done for emulator quota and period by commit ids 'b65dafa' and 'e051c482'. Being able to view/set the IOThread specific values is related to more recent changes adding global period (commmit id '4d92d58f') and global quota (commit id '55ecdae') definitions and qemu support (commit id '4e17ff79' and 'fbcbd1b2'). With a global setting though, if somehow the IOThread value in the cgroup hierarchy was set "outside of libvirt" to a value that is incompatible with the global value. Allowing control over IOThread specific values provides the capability to alter the IOThread values as necessary. 2016-08-03 Daniel P. Berrange Don't error when attaching security label of model "none" If you invoke virDomainLxcEnterSecurityLabel() on security model of "none" it will report an error. Logically a "none" security model should be treated as a no-op, so we should just return success immediately, instead of an error. qemu: only report errno in trace message on failure Avoid reporting a stale errno value when the syscall succeeds, instead always pass 0. 2016-08-02 Peter Krempa virsh: qemu-monitor-command: Don't print extra newline with --pretty The prettified JSON string already contains a newline so don't print another one. This allows to pipe the json output (in conjunction with the --quiet option) to files without having to truncate them afterwards. 2016-08-02 Peter Krempa tests: Make schema test fail on XML schema errors Failure to parse the schema file would not trigger a test suite failure. In addition to making the test fail it's necessary to split up the parsing of the schema file into a separate test. This is necessary as the XML validator uses libvirt errors to report problems parsing of the actual schema RNG needs to be split out into a separate function and called via virTestRun which has the infrastructure to report them. 2016-08-02 Peter Krempa remote: Fix indentation of the remote protocol structs check file 2016-08-02 John Ferlan qemu: Use the hostdev alias in qemuDomainAttachHostSCSIDevice error path https://bugzilla.redhat.com/show_bug.cgi?id=1289391 Rather than pass the whole drive string (which contained the alias), pass only the alias for the qemuMonitorDriveDel call in the error path when adding a host device in the monitor fails. 2016-08-02 John Ferlan qemu: Use qemuAliasFromHostdev When building the command line alias and for SCSI Host Device deletion, use the common API to build the alias qemu: Introduce qemuAliasFromHostdev Introduce a common API to generate the alias for a host device 2016-08-02 John Ferlan qemu: Add attempt to call qemuMonitorDriveDel for AttachSCSI failure path Completion of fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1336225 Similar to the other disk types, add the qemuMonitorDriveDel in the failure to add/hotplug a SCSI disk. 2016-08-02 John Ferlan qemu: Add attempt to call qemuMonitorDriveDel for USB failure path Partial fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1336225 Similar to the other disk types, add the qemuMonitorDriveDel in the failure to add/hotplug a USB. Added a couple of other formatting changes just to have a less cluttered look 2016-08-02 John Ferlan qemu: Make QEMU_DRIVE_HOST_PREFIX more private Move QEMU_DRIVE_HOST_PREFIX into the qemu_alias.c to dissuade future callers from using it. Create qemuAliasDiskDriveSkipPrefix in order to handle the current consumers that desire to check if an alias has the drive- prefix and "get beyond it" in order to get the disk alias. qemu: Use qemuAliasFromDisk to generate drive alias Rather than open code build the drive alias command in multiple places, use the helper to ensure consistency. 2016-08-02 John Ferlan qemu: Use qemuAliasFromDisk instead of qemuDeviceDriveHostAlias Since we already have a function that will generate the drivestr from the alias, let's use it and remove the qemuDeviceDriveHostAlias. Move the QEMU_DRIVE_HOST_PREFIX definition into qemu_alias.h Also alter qemuAliasFromDisk to use the QEMU_DRIVE_HOST_PREFIX instead of "drive-%s". 2016-08-02 John Ferlan qemu: Use qemuAliasFromDisk to generate drive alias The qemuDomainSnapshotCreateSingleDiskActive open coded generating the disk drive alias. Let's use the common function. qemu: Remove generation of drive alias from qcow passphrase backends Rather than pass the disks[i]->info.alias to qemuMonitorSetDrivePassphrase and then generate the "drive-%s" alias from that, let's use qemuAliasFromDisk prior to the call to generate the drive alias and then pass that along thus removing the need to generate the alias from the monitor code. qemu: Reorder qemuDomainAttachUSBMassStorageDevice failure path Modify the error/exit path to match what was done for Virtio and SCSI. If nothing else it'll have a consistent look'n'feel 2016-08-02 Jovanka Gulicoska event-test: support node device lifecycle event APIs node_device: Implement event queue in udev node_device: implement node device lifecycle event APIs remote: implement node device lifecycle event APIs test: implement node device lifecycle event APIs Also includes unittests for node device lifecycle events API conf: add node_device_event handling Add node device event handling infrastructure to node_device_event.[ch] 2016-08-02 Jovanka Gulicoska Introduce node device lifecycle event APIs Node device lifecycle event API entry points for registering and deregistering node deivce events, as well as types of events associated with node device. These entry points will be used for implementing asynchronous lifecycle events. Node device API: virConnectNodeDeviceEventRegisterAny virConnectNodeDeviceEventDeregisterAny virNodeDeviceEventLifecycleType which has events CREATED and DELETED 2016-08-02 Nikolay Shirokovskiy qemu: Set fake reboot flag to false when mode=agent As commit id 'e2b86f580' notes, when mode=agent possibly setting the fake reboot flag to true wouldn't be necessary; however, it doesn't "force" the issue by just ensuring the fake reboot is false, so this patch adds the explicit setting for the reboot path. More investigation and details can be found in commit id '8be502fd' as well as in the archives at: https://www.redhat.com/archives/libvir-list/2015-April/msg00715.html 2016-08-02 Nikolay Shirokovskiy qemu: Set fake reboot flag only in acpi mode for shutdown Conditional setting of the fake reboot flag should only happen for the acpi mode shutdown path; however, for the agent mode shutdown, the fake reboot should be cleared. This patch will essentially revert commit id '8be502fd', but adds an explicit setting of the flag to false when using mode=agent while also only conditionally setting the reboot flag if the guest went away. This also avoids an issue where a shutdown with reboot semantics is done from agent mode which sets the reboot flag followed by a shutdown from within the guest which would result in a reboot due to the fake reboot flag being set. The change will also properly handle the cases described in the following archive post: https://www.redhat.com/archives/libvir-list/2015-April/msg00715.html 2016-08-02 John Ferlan libxl: Fix broken build attach/detach controller device Commit id '44304c6eb' added the API libxlDomainAttachControllerDevice inside a conditional LIBXL_HAVE_PVUSB, but called that function outside the conditional in libxlDomainAttachDeviceLive. Similarly, the API libxlDomainDetachControllerDevice was added inside a conditional LIBXL_HAVE_PVUSB, but called outside the conditional in libxlDomainDetachDeviceLive. This patch adds the conditional LIBXL_HAVE_PVUSB around those two calls from within the switch. 2016-08-02 John Ferlan libxl: Fix broken build from libxlDomainCleanup Commit id '7d3b2eb5' missed a closing parenthesis on the ignore_value macro, causing the failure 2016-08-02 Erik Skultety rpc: virnetserver: Remove dead code checking the client limits Prior to commit 2737aaaf, we allowed every client to connect successfully, however, if accepting a client would eventually lead to an overcommit of the limits, we would disconnect it immediately with "Too many active clients, dropping connection from...". Recent changes refactored the code in a way, that it is not possible for the client-related callback to be dispatched and the client to be accepted if the limits wouldn't permit to do so, therefore a check if a connection should be dropped due to limits violation has become a dead code that could be removed. 2016-08-02 Erik Skultety admin: rpc: virnetserver: Fix updating of the client limits Commit 2737aaaf changed our policy for accepting new clients in a way, that instead of accepting new clients only to disconnect them immediately, since that would overcommit the limit, we temporarily disable polling for the dedicated file descriptor, so any new connection will queue on the socket. Commit 8b1f0469 then added the possibility to change the limits during runtime but it didn't re-enable polling for the previously disabled file descriptor, thus any new connection would still continue to queue on the socket. This patch forces an update of the services each time the limits were changed in some way. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1357776 2016-08-02 Erik Skultety rpc: virnetserver: Add code to CheckLimits to handle suspending of services So far, virNetServerCheckLimits was only used to possibly re-enable accepting new clients that might have previously been disabled due to client limits violation (max_clients, max_anonymous_clients). This patch refactors virNetServerAddClient, which is currently the only place where the services get disabled, in order to use the virNetServerCheckLimits helper instead of checking the limits by itself. rpc: virnetserver: Move virNetServerCheckLimits which is static up in the file Since virNetServerAddClient checks for the limits in order to temporarily suspend the services, thus not accepting any more clients, there is no reason why virNetServerCheckLimits, which is only responsible for re-enabling previously disabled services according to the limits, could not do both. To be able to do that however, it needs to be moved up in the file since it's static (and because it's just a helper and there's only one caller it should remain static). rpc: virnetserver: Rename ClientSetProcessingControls to ClientSetLimits The original naming was just a leftover that should have been fixed in commit 8b1f0469. 2016-08-02 Cédric Bosdonnat libxl: add hooks support Introduce libxl hook and use it for start, prepare, started, stop, stopped, migrate events. 2016-08-02 Cédric Bosdonnat libxl: fix segfault in libxlReconnectDomain In case of error, libxlReconnectDomain may call virDomainObjListRemoveLocked. However it has no local reference on the domain object, leading to segfault. Get a reference to the domain object at the start of the function and release it at the end to avoid problems. This commit also factorizes code between the error and normal ends. 2016-08-02 Cédric Bosdonnat libxl: add a flag to mark guests as tainted by a hook The migrate hook will affect the migrated guest definition. Allow these domains be marked as tainted in the libxl driver. 2016-08-02 Chunyan Liu qemuDomainDeviceDefPostParse: add USB controller model check To sync with virDomainControllerModelUSB, we add two models in qemuControllerModelUSB 'qusb1' and 'qusb2', but those models are not supported in qemu driver. So add check in device post parse to report errors if 'qusb1' and 'qusb2' are specified. xenconfig: add conversion of usb controller config to and from xml libxl configuration files conversion can now handle USB controllers. When parting libxl config file, USB controllers with type PV are ignored as those aren't handled. libxl: check available controller and port when hotplugging USB device When hotplugging a USB device, check if there is an available controller and port, if not, automatically create a USB controller of version 2.0 and 8 ports. 2016-08-02 Chunyan Liu libxl: support usb controller hotplug Support USB controller hot-plug and hot-unplug. #virsh attach-device dom usbctrl.xml #virsh detach-device dom usbctrl.xml usbctrl.xml example: 2016-08-02 Chunyan Liu libxl: support USB controllers in creation time To support USB Controller in xen guest domains, just add USB controller in domain config xml as following: 2016-08-02 Chunyan Liu extend usb controller model to support xen pvusb According to libxl implementation, it supports pvusb controller of version 1.1 and version 2.0, and it supports two types of backend, 'pvusb' (dom0 backend) and 'qusb' (qemu backend). But currently pvusb backend is not checked in yet. To match libxl support, extend usb controller schema to support two more models: qusb1 (qusb, version 1.1) and 'qusb2' (qusb version 2.0). 2016-08-02 Peter Krempa virsh: qemu-monitor-command: Simplify control flow Construct the query string by using virBufferTrim rather than having to remember to add a space and simplify cleanup path. virsh: qemu-monitor-command: Use macro for exclusive options qemu: cap: Refactor access to array in virQEMUCapsProbeQMPMachineTypes Use a temporary pointer rather than always recalculating the index in a very verbose way. qemu: caps: Sanitize storage of machine type related data Add a structure to store the data and use a single array of the structures rather than having 3 separate arrays with shared indexes. qemu: capabilities: Drop unused function virQEMUCapsGetMachineTypes 2016-08-02 Martin Kletzander storage: Clean up volume wiping Let's cleanly differentiate what wiping a volume does for ploop and other volumes so it's more readable what is done for each one instead of branching out multiple times in different parts of the same function. storage: Use path instead of volume as an argument Some functions use volume specification merely to use the target path from it. Let's change it to pass the path only so that it can be used for other files than just volumes. storage: Move functions around This is done in order to call them in next patches from each other and definitions would be missing otherwise. 2016-08-02 Martin Kletzander qemu: Fix domain state after reset When reset was called from a domain that crashed we didn't change the crashed state into a paused one which could confuse users. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1269575 2016-08-02 Martin Kletzander virsh: Report error when explicit connection fails Commit 0c56d9431839 forgot to return false in the cmdConnect command after the clean up made there. Before (assuming you don't have uri alias for 'asdf'): $ virsh connect asdf error: failed to connect to the hypervisor $ echo $? 0 After (with the same assumption): $ virsh connect asdf error: failed to connect to the hypervisor error: no connection driver available for asdf $ echo $? 1 Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1356461 2016-08-02 Martin Kletzander qemu: Fix support for startupPolicy with volume/pool disks Until now we simply errored out when the translation from pool+volume failed. However, we should instead check whether that disk is needed or not since there is an option for that. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1168453 2016-08-02 Martin Kletzander qemu: Remove unnecessary label and its only reference qemu: Make qemuDomainCheckDiskStartupPolicy self-contained There is an error reset following the function and check for startupPolicy before that. Let's reflect those things inside that function so that future code doesn't have to be that complex. 2016-08-02 Michal Privoznik Post-release version bump to 2.2.0 2016-08-02 Yuri Chornoivan Fix unbalanced quotation marks 2016-08-02 Daniel Veillard Release of libvirt-2.1.0 * docs/news.html.in: updated for release * po/*.po*: regenerated Revert "Fix unbalanced quotation marks" This reverts commit 6a40801186d1fcba38360f58659f898b67f08a01. release of 2.1.0 need to go first 2016-08-02 Yuri Chornoivan Fix unbalanced quotation marks 2016-08-02 Erik Skultety daemon: sasl: Don't forget to save SASL username to client's identity Once the SASL authentication process has successfully passed, we should also save the SASL username used to client's identity, so that when a client like virt-admin tries to obtain it, the server will actually format the username to the response data. admin: Retrieve the SASL context for both local and remote connection When commit 4a0e9108 added a support for client information retrieval, it made the API return SASL identity info only for clients connected remotely, yet SASL can be happily used with UNIX sockets as well. 2016-08-02 Martin Kletzander storage: Document wiping formatted volume types When wiping a volume we just rewrite all the data of the volume, not only the content. Since format gets overridden, we need to recreate the volume. However we can't do that for every possible format out there. Since it was only coded for the ploop volume type, let's document what might be the consequences instead of forbidding it for every other format out there. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=868771 2016-08-01 Boris Fiuczynski qemu: remove panic dev models s390 and pseries when migrating The panic devices with models s390 and pseries are autogenerated. For backwards compatibility reasons the devices are to be removed when migrating. 2016-08-01 Michal Privoznik wireshark: Drop glib dependency The only function that we currently use from glib is g_sprintf(). That's a very big gun for such small target. Not only that, but we've silently relied on wireshark dragging in the glib. Replace the g_sprintf() with plain sprinf() so that we can drop the glib dependency. 2016-08-01 Nikolay Shirokovskiy schema: fix resolved interfaces of network type This patch reflects cases when element and its subelement for network type are formated based on actual type resolved from referenced network instead of original one. networkAllocateActualDevice and virDomainActualNetDefContentsFormat are taken as reference. schema: add missed alias element to memory device 2016-08-01 Sascha Silbe qemu: fix domain id after domainCreateWithFlags() Ever since virDomainCreateWithFlags() was introduced by de3aadaa [drivers: add virDomainCreateWithFlags if virDomainCreate exists], the domain ID retrieved with virDomainGetID() was incorrect for several drivers after virDomainCreateWithFlags() was called. The API consumer had to look up the domain anew to retrieve the correct ID. For the ESX driver, this was fixed in 6139b274 [esx: Update ID after starting a domain]. For the openvz driver, it was fixed in fd81a097 [openvzDomainCreateWithFlags: set domain id to the correct value]. The test driver, the OpenNebula driver (removed in the meantime) and the vbox driver were already updating the domain ID correctly in domainCreate(). Copy over the ID in qemuDomainCreateWithFlags() to fix this for the qemu driver, too. Fixes: de3aadaa ("drivers: add virDomainCreateWithFlags if virDomainCreate exists") Reported-by: Marc Hartmayer Tested-by: Marc Hartmayer Reviewed-by: Marc Hartmayer 2016-08-01 Peter Krempa tests: qemu: Don't leak security manager object ==2064442== 200 (88 direct, 112 indirect) bytes in 1 blocks are definitely lost in loss record 54 of 73 ==2064442== at 0x4C2E0F0: calloc (vg_replace_malloc.c:711) ==2064442== by 0x18E75B80: virAllocVar (viralloc.c:560) ==2064442== by 0x18EC43B0: virObjectNew (virobject.c:193) ==2064442== by 0x18EC476E: virObjectLockableNew (virobject.c:219) ==2064442== by 0x1906BC73: virSecurityManagerNewDriver (security_manager.c:93) ==2064442== by 0x1906C076: virSecurityManagerNewStack (security_manager.c:115) ==2064442== by 0x43CC39: qemuTestDriverInit (testutilsqemu.c:548) ==2064442== by 0x4337ED: mymain (qemumonitorjsontest.c:2440) ==2064442== by 0x43BABE: virTestMain (testutils.c:982) ==2064442== by 0x43A490: main (qemumonitorjsontest.c:2558) 2016-07-29 Michal Privoznik conf: Catch invalid memory model earlier Consider the following XML snippet: 523264 0 Whats wrong you ask? The @model attribute. This should result in an error thrown into users faces during virDomainDefine phase. Except it doesn't. The XML validation catches this error, but if users chose to ignore that, they will end up with invalid XML. Well, they won't be able to start the machine - that's when error is produced currently. But it would be nice if we could catch the error like this earlier. 2016-07-29 Erik Skultety admin: Fix default uri config option name s/admin_uri_default/uri_default The original name 'admin_uri_default' was introduced to our code by commit dbecb87f. However, at that time we already had a separate config file for admin library but the commit mentioned above didn't properly adjust the config's option name. The result is that when we're loading the config, we check a non-existent config option (there's not much to do with the URIs anyway, since we only allow local connection). Additionally, virt-admin's man page documents, that the default URI can be altered by setting admin_uri_default option. So the fix proposed by this patch leaves the libvirt-admin.conf as is and adjusts the naming in the code as well as in the virt-admin's man page. 2016-07-28 Michal Privoznik wireshark: Adapt to dissector function header change In wireshark commit bbdd89b9 (contained in 2.1.0 release) they have changed prototype of dissector function. Now it returns number of bytes consumed by the dissector, and can get a pointer to user specified data (which we don't use). 2016-07-28 John Ferlan qemu: Need to free fileprops in error path The virJSONValueObjectCreate only consumes the object on success, so on failure we must free - from commit id 'f4441017' (found by Coverity). 2016-07-28 John Ferlan iscsi: Establish connection to target via static target login https://bugzilla.redhat.com/show_bug.cgi?id=1356436 Commit id '56057900' altered the discovery of iSCSI node targets by using the "--op nonpersistent". This caused issues for clean environments or if by chance a "-m node -o delete" was executed. Since each iSCSI Storage Pool has the required iSCSI target path, use that and the virISCSINodeNew API in order to generate the iSCSI node record. 2016-07-28 John Ferlan util: Introduce virISCSINodeNew https://bugzilla.redhat.com/show_bug.cgi?id=1356436 According to RFC 3721 (https://www.ietf.org/rfc/rfc3721.txt), there are two ways to "discover" targets in/for the iSCSI environment. Discovery is the process which allows the initiator to find the targets to which it has access and at least one address at which each target may be accessed. The method currently implemented in libvirt using the virISCSIScanTargets API is known as "SendTargets" discovery. This method is more useful when the target IP Address and TCP port information are available, e.g. in libvirt terms the "portal". It returns a list of targets for the portal. From that list, the target can be found. This operation can also fill an iSCSI node table into which iSCSI logins may occur. Commit id '56057900' altered that filling by adding the "--op nonpersistent" since it was not necessarily desired to perform that for non libvirt related targets. The second method is "Static Configuration". This method not only needs the IP Address and TCP port (e.g. portal), but also the iSCSI target name. In libvirt terms this would be the device path field from the iSCSI pool XML. This patch implements the second methodology using that required device path as the targetname. 2016-07-28 Erik Skultety tools: Make use of the correct environment variables Since commit 834c5720 which extracted the generic functionality out of virsh and made it available for other clients like virt-admin to make use of it, it also introduced a bug when it renamed the original VIRSH_ environment variables to VSH_ variables. Virt-admin of course suffers from the same bug, so this patch modifies the generic module vsh.c to construct the correct name for environment variables of each client from information it has. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1357363 2016-07-28 Erik Skultety vsh: Make vshInitDebug return int instead of void Well, the reason behind this change is that if the function is extended in some way that e.g. would involve allocation we do not have a way of telling it to the caller. More specifically, vshInitDebug only relies on some hardcoded environment variables (by a mistake) that aren't documented anywhere so neither virsh's nor virt-admin's documented environment variables take effect. One possible solution would be duplicate the code for each CLI client or leave the method be generic and provide means that it could figure out, which client called it, thus initializing the proper environment variables but that could involve operations that might as well fail in certain circumstances and the caller should know that an error occurred. 2016-07-28 Daniel P. Berrange Refresh translations from zanata 2016-07-28 Michal Privoznik vshReadlineParse: Drop some unused variables My compiler identified some variables that were set, but never actually used. For instance, opts_required, and data_acomplete. vshCmddefGetOption: Change type of opt_index This function tries to look up desired option for a given parsed command. Upon successful return it also stores option position into passed *opt_index. Now, this variable is type of int, even though it is never ever used to store negative value. Moreover, the variable is set from a local variable which is type of size_t. 2016-07-28 Daniel P. Berrange libvirt.spec.in: fix indentation in previous commit 2016-07-28 Daniel P. Berrange Fix RPM spec for wireshark on Fedora < 24 This previous commit commit cd9fcc8be7dcb9126d70f744ce54b0b742eeefb8 Author: Michal Privoznik Date: Wed Jul 27 16:58:32 2016 +0200 libvirt.spec.in: Adapt to newest wireshark plugindir Adapted the libvirt spec for wireshark >= 2.1.0 but this ignored the fact that we enable wireshark from Fedora 21 and 2.1.0 was only added in Fedora 24 2016-07-27 Daniel P. Berrange storage: remove "luks" storage volume type The current LUKS support has a "luks" volume type which has a "luks" encryption format. This partially makes sense if you consider the QEMU shorthand syntax only requires you to specify a format=luks, and it'll automagically uses "raw" as the next level driver. QEMU will however let you override the "raw" with any other driver it supports (vmdk, qcow, rbd, iscsi, etc, etc) IOW the intention though is that the "luks" encryption format is applied to all disk formats (whether raw, qcow2, rbd, gluster or whatever). As such it doesn't make much sense for libvirt to say the volume type is "luks" - we should be saying that it is a "raw" file, but with "luks" encryption applied. IOW, when creating a storage volume we should use this XML demo.raw 5368709120 and when configuring a guest disk we should use This commit thus removes the "luks" storage volume type added in commit 318ebb36f1027b3357a32d6f781bd77d7a9043fd Author: John Ferlan Date: Tue Jun 21 12:59:54 2016 -0400 util: Add 'luks' to the FileTypeInfo The storage file probing code is modified so that it can probe the actual encryption formats explicitly, rather than merely probing existance of encryption and letting the storage driver guess the format. The rest of the code is then adapted to deal with VIR_STORAGE_FILE_RAW w/ VIR_STORAGE_ENCRYPTION_FORMAT_LUKS instead of just VIR_STORAGE_FILE_LUKS. The commit mentioned above was included in libvirt v2.0.0. So when querying volume XML this will be a change in behaviour vs the 2.0.0 release - it'll report 'raw' instead of 'luks' for the volume format, but still report 'luks' for encryption format. I think this change is OK because the storage driver did not include any support for creating volumes, nor starting guets with luks volumes in v2.0.0 - that only since then. Clearly if we change this we must do it before v2.1.0 though. 2016-07-27 Daniel P. Berrange virstoragefile: refactor virStorageFileMatchesNNN methods Refactor the virStorageFileMatchesNNN methods so that they don't take a struct FileFormatInfo parameter, but instead get the actual raw dat items they needs. This will facilitate reuse in other contexts. 2016-07-27 Michal Privoznik libvirt.spec.in: Adapt to newest wireshark plugindir In the old days, when wireshark plugin was introduced it was installed under /usr/lib64/wireshark/plugins/$VERSION/ while with wireshark-2.1.0 this path has changed just to /usr/lib64/wireshark/plugins. We should teach our spec file about this change. 2016-07-27 Michal Privoznik virt-wireshark: Properly substract wireshark prefix So, when building wireshark plugin, we get the plugindir variable from the wireshark.pc as well as prefix. Then we replace the prefix in the plugindir with our own prefix where libvirt is building to: plugindir="${prefix}${plugindir#ws_prefix}" However, as you can see, there's '$' missing in front of the ws_prefix variable. This results in the mangled plugindir, for instance like this: plugindir='/usr/usr/lib64/wireshark/plugins' 2016-07-27 Derbyshev Dmitry qemu: return balloon statistics when all domain statistics reported To collect all balloon statistics for all guests it was necessary to make several libvirt requests. Now it's possible to get all balloon statiscs via single connectGetAllDomainStats call. qemu: split qemuDomainMemoryStats into internal and external functions Is necessary to call it from other contexts, such as qemuDomainGetStatsBalloon. 2016-07-27 Pavel Hrdina qemu: fix domain memory 'last-update' timestamp This fixes commit 200a40f9 which introduced 'last-update' timestamp. qemu: fix domain memory 'usable' stat This fixes commit 65bf0446 which introduced 'usable' stat. 2016-07-27 Derbyshev Dmitry virsh: Add balloon stats description to .pod Description for existing balloon stats was missing for dommemstat. 2016-07-27 Erik Skultety virt-admin.pod: Remove a statement about remote access to the daemon There's been a forgotten fragment (copy-paste error probably) in the virt-admin's man page referring the reader to our web page on how to construct URIs in case of remote access, which sort of implies that we support it which we don't at the moment, so better remove that. 2016-07-27 Prasanna Kumar Kalever qemu: command: Add support for multi-host gluster disks To allow using failover with gluster it's necessary to specify multiple volume hosts. Add support for starting qemu with such configurations. 2016-07-27 Peter Krempa qemu: command: Add infrastructure for object specified disk sources To allow richer definitions of disk sources add infrastructure that will allow to register functionst generating a JSON object based definition. This infrastructure will then convert the definition to the proper command line syntax and use it in cases where it's necessary. This will allow to keep legacy definitions for back-compat when possible and use the new definitions for the configurations requiring them. 2016-07-27 Peter Krempa util: qemu: Add support for numbered array members Add support for converting objects nested in arrays with a numbering discriminator on the command line. This syntax is used for the object-based specification of disk source properties. 2016-07-27 Peter Krempa storage: gluster: Support multiple hosts in backend functions As gluster natively supports multiple hosts for failover reasons we can easily add the support to the storage driver code in libvirt. Extract the code setting an individual host into a separate function and call them in a loop. The new code also tries to keep the debug log entries sane. 2016-07-27 Peter Krempa qemu: command: Refactor code extracted to qemuBuildDriveSourceStr Avoid a large block by tweaking the condition skipping empty drives and split up the switch containing two branches having different purpose. qemu: command: Extract drive source command line formatter The disk source formatting code grew rather ugly and complex and it will get worse. Extract it into a separated function to contain the mess. qemu: command: Split out network disk URI building Extract the code so that it can be called from multiple places. This also removes a tricky fallthrough in the large switch in qemuBuildNetworkDriveStr. qemu: command: Rename qemuBuildNetworkDriveURI to qemuBuildNetworkDriveStr The function builds also non-uri strings for the various protocols. util: storage: Add JSON backing volume parser for 'ssh' protocol util: storage: Add 'ssh' network storage protocol Allow using 'ssh' protocol in backing chains and later for disks themselves. util: storage: Add JSON backing store parser for 'sheepdog' protocol util: storage: Add JSON backing volume parser for 'nbd' protocol util: storage: Add json pseudo protocol support for iSCSI volumes iSCSI is a bit odd in this aspect since it only supports URIs but using the 'filename' property and does not have any alternative syntax. util: storage: Add json pseudo protocol support for gluster volumes Along with the legacy URI based syntax add support for the brand-new fully object based syntax. util: storage: Add support for URI based backing volumes in qemu's JSON pseudo-protocol http(s), ftp(s) and tftp use URIs for volume definitions in the JSON pseudo protocol so it's pretty straightforward to add support for them. util: storage: Add support for host device backing specified via JSON JSON pseudo protocol for qemu allows to explicitly specify devices. Add convertor to the internal type. 2016-07-27 Peter Krempa util: storage: Add parser for qemu's json backing pseudo-protocol Add a modular parser that will allow to parse 'json' backing definitions that are supported by qemu. The initial implementation adds support for the 'file' driver. Due to the approach qemu took to implement the JSON backing strings it's possible to specify them in two approaches. The object approach: json:{ "file" : { "driver":"file", "filename":"/path/to/file" } } And a partially flattened approach: json:{"file.driver":"file" "file.filename":"/path/to/file" } Both of the above are supported by qemu and by the code added in this commit. The current implementation de-flattens the first level ('file.') if possible and required. Other handling may be added later but currently only one level was possible anyways. 2016-07-27 Peter Krempa util: json: Make first argument of virJSONValueCopy const It's just read. 2016-07-27 Shivaprasad G Bhat qemu: Adjust the cur_ballon on coldplug/unplug of dimms The cur_balloon also increases/decreases with dimm hotplug/unplug. To be consistent, adjust the value for coldplug too. This was inconsistently taken care when cur_ballon != memory to begin with. The patch fixes it irrespective of that. 2016-07-27 Erik Skultety virconf: Fix config file path construction Since commit c4bdff19, the path to the configuration file has been constructed in the following manner: - if no config filename was passed to virConfLoadConfigPath, libvirt.conf was used as default - otherwise the filename was concatenated with "/libvirt/libvirt%s%s.conf" which in admin case resulted in "libvirt-libvirt-admin.conf.conf". Obviously, this non-existent config led to ignoring all user settings in libvirt-admin.conf. This patch requires the config filename to be always provided as an argument with the concatenation being simplified. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1357364 2016-07-27 Ramon Medeiros docs: Add Kimchi as Web Application Kimchi is a open-source interface to kvm. It runs with HTML5, simple and easy to manage kvm guests. 2016-07-27 Peter Krempa util: qemu: Don't generate any extra commas in virQEMUBuildCommandLineJSON The function would generate a leading comma. Let the callers properly add commas by formatting the commas at the end and trimming the trailing one. 2016-07-27 Peter Krempa util: qemu: Allow for different approaches to format JSON arrays For use with memory hotplug virQEMUBuildCommandLineJSONRecurse attempted to format JSON arrays as bitmap on the command line. Make the formatter function configurable so that it can be reused with different syntaxes of arrays such as numbered arrays for use with disk sources. This patch extracts the code and adds a parameter for the function that will allow to plug in different formatters. 2016-07-27 Peter Krempa util: qemu: Allow nested objects in JSON -> commandline generator Move the iterator of objects to the recursive function so that nested objects are supported by flattening the structure with '.' delimiters. util: qemu: Add support for user-passed strings in JSON->commandline Until now the JSON->commandline convertor was used only for objects created by qemu. To allow reusing it with disk formatter we'll need to escape ',' as usual in qemu commandlines. 2016-07-27 Peter Krempa util: qemu: Add wrapper for JSON -> commandline conversion Refactor the command line generator by adding a wrapper (with documentation) that will handle the outermost object iteration. This patch also renames the functions and tweaks the error message for nested arrays to be more universal. The new function is then reused to simplify qemucommandutiltest. 2016-07-27 Peter Krempa util: json: Make first argument of virJSONValueObjectForeachKeyValue const The iterator function (second argument) already requires that the object is handled as 'const' thus we won't modify the object itself. 2016-07-27 Peter Krempa tests: Add testing of backing store string parser As we already test that the extraction of the backing store string works well additional tests for the backing store string parser can be made simpler. Export virStorageSourceNewFromBackingAbsolute and use it to parse the backing store strings, format them using virDomainDiskSourceFormat and match them against expected XMLs. 2016-07-27 Peter Krempa tests: qemuxml2xml: Avoid crash when processing an XML that fails to parse Failure to parse a XML that was not supposed to fail would result into a crash in the test suite as the vcpu bitmap would not be filled prior to the active XML->XML test. Skip formatting of the vcpu snippet in the fake status XML formatter in such case to avoid the crash. The test would fail anyways. 2016-07-27 John Ferlan Remove unnecessary virDomainDefClearDeviceAliases Nothing in the code path after the removed call has needs/uses the alias anyway (as would be the case for command line building or talking to monitor). The alias is VIR_FREE'd in virDomainDeviceInfoClear which is called for any device that needs/uses an alias via virDomainDeviceDefFree or virDomainDefFree as well as during virDomainDeviceInfoFree for host devices. For persistent domains, the domain definition (including aliases) gets freed a few screens later when it's replaced with newDef. For transient domains, the definition is freed/unref'd along with the virDomainObj a few moments later. 2016-07-27 John Ferlan conf: Remove CCW,PCI clear address helpers Since commit id 'fb06350' these are no longer called, so remove them 2016-07-26 Joao Martins libxl: implement virDomainBlockStats Introduce initial support for domainBlockStats API call that allow us to query block device statistics. OpenStack nova uses this API call to query block statistics, alongside virDomainMemoryStats and virDomainInterfaceStats. Note that this patch only introduces it for VBD for starters. QDisk would come in a separate patch series. A new statistics data structure is introduced to fit common statistics among others specific to the underlying block backends. For the VBD statistics on linux these are exported via sysfs on the path: "/sys/bus/xen-backend/devices/vbd--/statistics" To calculate the block devno libxlDiskPathToID is introduced. Each backend implements its own function to extract statistics, allowing support for multiple backends and different platforms. VBD stats are exposed in reqs and number of sectors from blkback, and it's up to us to convert it to sector sizes. The sector size is gathered through xenstore in the device backend entry "physical-sector-size". BlockStatsFlags variant is also implemented which has the added benefit of getting the number of flush requests. 2016-07-26 Henning Schild qemu: rename QEMU_CAPS_MLOCK to QEMU_CAPS_REALTIME_MLOCK Purely cosmetic change to be consistent with the other names. 2016-07-26 Jovanka Gulicoska conf: events: handle NULL uuid Not all objects have a uuid, such as nodedevs. When we add events support for them, NULL will be passed here, so handle it. 2016-07-26 Anton Khramov network: Added hook for network modification event Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1181539 2016-07-26 Derbyshev Dmitry qemu: expand domain memory statistics with 'last-update' timestamp QEMU reports timestamp along with other memory statistics, but this information is not saved into domain statistics. It could be useful to determine if the data reported is fresh or not. Balloon statistics are not reported in hrf, so no modifications are made in qemu_monitor_text.c. qemu: expand domain memory statistics with 'usable' 'memtotal' in virtio drivers and qemu corresponds to 'available' in libvirt. Because of that, 'stat-available-memory' is renamed into 'usable'. Balloon statistics are not reported in hrf, so no modifications are made in qemu_monitor_text.c. 2016-07-26 Tomasz Flendrich qemu: remove ccwaddrs caching Dropping the caching of ccw address set. The cached set is not required anymore, because the set is now being recalculated from the domain definition on demand, so the cache can be deleted. qemu_hotplug: generate ccw address list on demand Dropping the caching of ccw address set. Instead of using the cached address set, functions in qemu_hotplug.c now recalculate it on demand. 2016-07-26 Tomasz Flendrich Add qemuDomainCCWAddrSetCreateFromDomain The address sets (pci, ccw, virtio serial) are currently cached in qemu private data, but all the information required to recreate these sets is in the domain definition. Therefore I am removing the redundant data and adding a way to recalculate these sets. Add a function that calculates the ccw address set from the domain definition. 2016-07-26 Tomasz Flendrich qemu: remove vioserialaddrs caching Dropping the caching of virtio serial address set. The cached set is not required anymore, because the set is now being recalculated from the domain definition on demand, so the cache can be deleted. Credit goes to Cole Robinson. 2016-07-26 Tomasz Flendrich qemu_hotplug: generate vioserial address list on demand Dropping the caching of virtio serial address set. Instead of using the cached address set, a function in qemu_hotplug.c now recalculates it on demand. Credit goes to Cole Robinson. 2016-07-26 Tomasz Flendrich add virDomainVirtioSerialAddrSetCreateFromDomain The address sets (pci, ccw, virtio serial) are currently cached in qemu private data, but all the information required to recreate these sets is in the domain definition. Therefore I am removing the redundant data and adding a way to recalculate these sets. Add a function that calculates the virtio serial address set from the domain definition. Credit goes to Cole Robinson. 2016-07-26 Ján Tomko cgroup: drop INSERT_ELEMENT usage virCgroupPartitionEscape Use virAsprintf to prepend an underscore to make the code more readable. 2016-07-26 Andrea Bolognani util: systemd: Define MSG_NOSIGNAL if needed The symbol being missing has been reported as causing build failures on OS X. If it's not already defined, define it to zero so that it won't have any effect. 2016-07-25 Pavel Hrdina qemu_monitor: search memballoon QOM device path using alias Commit 4a585a88 introduced searching QOM device path by alias, let's use it for memballoon too. This may speedup the search because in most cases we will find the correct QOM device path directly by using alias without the need for the recursion code. 2016-07-25 Pavel Hrdina qemu_monitor_json: add support to search QOM device path by device alias Commit ce745914 introduced detection of actual video ram sizes to fix migration if QEMU decide to modify the values provided by libvirt. This works perfectly for domains with number of video devices up to two. If there are more than two video devices in the guest all the secondary devices in the XML will have the same memory values. This is because our current code search for QOM device path only by the device type name and all the secondary video devices has the same name "qxl". This patch introduces a new search function that will try to search a QOM device path using also device's alias if the alias is available. After that it will fallback to the old recursive code if the alias search found no results. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1358728 2016-07-25 Tomasz Flendrich Split qemuDomainDetachDeviceFlags in two Previously, qemuDomainDetachDeviceFlags was doing two things: handling the job and detaching devices. Now the second part is in a new function. Narrow down a parameter in qemuDomainDetachDeviceFlags This will make splitting up qemuDomainDetachDeviceFlags into two functions easier. 2016-07-25 Tomasz Flendrich Split qemuDomainAttachDeviceFlags in two Previously, qemuDomainAttachDeviceFlags was doing two things: handling the job and attaching devices. Now the second part is in a new function. This change is required to make it possible to test more complex device attachment situations, like attaching a device to both config and live at once. 2016-07-25 Tomasz Flendrich qemu: Remove an unnecessary variables qemuCaps is no longer used anywhere in these functions, so it can be deleted. Change parameters to qemuDomainAttachDeviceLive We want to be able to pass a NULL instead of the connection and use this function in tests. To achieve this, the virConnectPtr is passed instead of virDomainPtr, and the driver is a new separate parameter. 2016-07-25 Tomasz Flendrich qemuhotplugtest: Add tests for ccw devices There's a plan to rework the address handling, so testcases that verify hotplugging ccw devices will help in avoiding regression. In this commit, some files are duplicated because of the way qemuhotplug.c calculates the expected xml filenames. I plan on changing that to explicitly stating the basis domain xml, the device xml, and the expected xml. 2016-07-24 Martin Kletzander lxc: Don't crash by forgetting to ref transient domains So commit 306b3a8504 tried mimicking behaviour of commit 540c339a25, but added a virObjectRef(vm) only after virDomainObjListAdd() in lxcDomainDefineXMLFlags() and not in lxcDomainCreateXMLWithFiles(). That way undefining a domain that was started with different XML than defined will leave the domain object in a state with not enough references to then remove it. Hence any lxcDomainDestroyFlags() called afterwards crashes the daemon. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1351057 2016-07-22 Guido Günther virt-aa-helper: Make help output match option name 2016-07-22 Bjoern Walk qemu: hotplug: fix changeable media ejection Since return code is checked globally at the end of the function, let's make sure that we set it correctly at any point. This fixes a regression introduced in commit 0aa19f35 where the first command to eject changeable media would fail unconditionally. Reviewed-by: Boris Fiuczynski 2016-07-22 Katerina Koukiou lxc: make container's init process session leader This patch forces container's init process, to become a session leader, that is its session ID is made the same as its process ID. That might seem unnecessary in general, but if we want to checkpoint a container with CRIU, which is needed for container migration, we must ensure that the SID of each process inside the container points to a process that lives in the same PID namespace as the container. Therefore, we force that the session leader is the init. 2016-07-21 Ján Tomko Auto-add one hub if there are too many USB devices When parsing a command line with USB devices that have no address specified, QEMU automatically adds a USB hub if the device would fill up all the available USB ports. To help most of the users, add one hub if there are more USB devices than available ports. For wilder configurations, expect the user to provide us with more hubs and/or controllers. 2016-07-21 Ján Tomko Assign addresses on USB device hotplug USB disks, redirected devices, host devices and serial devices are supported. 2016-07-21 Ján Tomko Assign addresses to USB devices Automatically assign addresses to USB devices. Just like reserving, this is only done for newly defined domains. https://bugzilla.redhat.com/show_bug.cgi?id=1215968 2016-07-21 Ján Tomko Add tests for USB address assignment Introduce tests with the ich9, xhci and the default (piix3) usb controller to demonstrate the effect of the next patch. Reserve existing USB addresses Check if they fit on the USB controllers the domain has, and error out if two devices try to use the same address. Add functions for adding USB hubs to addrs Walk through all the usb hubs in the domain definition that have a USB address specified, create the corresponding structures in the virDomainUSBAddressSet and mark the port it occupies as used. Add functions for adding USB controllers to addrs Walk through all the usb controllers in the domain definition and create the corresponding structures in the virDomainUSBAddressSet. 2016-07-21 Ján Tomko Introduce virDomainUSBAddressSet A new type to track USB addresses. Every is represented by an object of type virDomainUSBAddressHub located at buses[i]. Each of these hubs has up to 'nports' ports. If a port is occupied, it has the corresponding bit set in the 'ports' bitmap, e.g. port 1 would have the 0th bit set. If there is a hub on this port, then hubs[i] will point to this hub. 2016-07-20 Nikolay Shirokovskiy vz: handle gracefully races on undefining domain Undefine procedure drops domain lock while waiting for detaching disks vz sdk call. Meanwhile vz sdk event domain-config-changed arrives, its handler finds domain and is blocked waiting for job condition. After undefine API call finishes event processing procedes and tries to refreshes domain config thru existing vz sdk domain handle. Domain does not exists anymore and event processing fails. Everything is fine we just don't want to see error message in log for this particular case. Fortunately domain has flag that domain is removed from list. This also imply that vz sdk domain is also undefined. Thus if we check for this flag right after domain is locked again on accuiring job condition we gracefully handle this situation. Actually the race can happen in other situations too. Any time we wait for job condition in mutualy exclusive job in time when we acquire it vz sdk domain can cease to exist. So instead of general internal error we can return domain not found which is easier to handle. We don't need to patch other places in mutually exclusive jobs where domain lock is dropped as if job is started domain can't be undefine by mutually exclusive undefine job. The code of this patch is quite similar to qemu driver checks for is domain is active after acquiring a job. The difference only while qemu domain is operational while process is active vz domain is operational while domain exists. 2016-07-20 Nikolay Shirokovskiy vz: make vz driver more responsive Current vz driver implementation is not usable when it comes to long runnig operations. Migration or saving a domain blocks all other operations even query ones which are expecteted to be available. This patch addresses this problem. All vz driver API calls fall into next 3 groups: 1. only query domain cache (virDomainObj, vz cache statistic) examples are vzDomainGetState, vzDomainGetXMLDesc etc. 2. use thread shared sdkdom object examples are vzDomainSetMemoryFlags, vzDomainAttachDevice etc. 3. use no thread shared sdkdom object nor domain cache examples are vzDomainSnapshotListNames, vzDomainSnapshotGetXMLDesc etc API calls from group 1 don't need to be changed as they hold domain lock only for short period of time. These calls [1] are easily distinguished. They query domain object thru libvirt common code or query vz sdk statistics handle thru vz sdk sync operations. vzDomainInterfaceStats is the only exception. It uses sdkdom object to convert interface name to its vz sdk stack index which could not be saved in domain cache. Interface statistics is available thru this stack index as a key rather than name. As a result we can have accidental 'not known interface' errors on quering intrerface stats. The reason is that in the process of updating domain configuration we drop all devices and then recreate them again in sdkdom object and domain lock can be dropped meanwhile (to remove networks for existing bridged interfaces and(or) (re)create new ones). We can fix this by changing the way we support bridged interfaces or by reordering operations and changing bridged networks beforehand. Anyway this is better than moving this API call into 2 group and making it an exclusive job. As to API calls from group 2, first thread shared sdkdom object needs to be explained. vz sdk has only one handle for a given domain, thus threads need exclusive access to operate on it. These calls are fixed to drop and reacquire domain lock on any lengthy operations - namely waiting the result of async vz sdk operation. As lock is dropped we need to take extra reference to domain object if it is not taken already as domain object can be deleted from list while lock is dropped. As this operations use thread shared sdkdom object, the simplest way to make calls from group 2 be consistent to each other is to make them mutually exclusive. This is done by taking/releasing job condition thru calling correspondent job routine. This approach makes group 1 and group 2 calls consistent to each other too. Not all calls of group 2 change the domain cache but those that do update it thru prlsdkUpdateDomain which holds the lock thoughout the update. API calls from group [2] are easily distinguished too. They use beginEdit/commit to change domain configuration (vzDomainSetMemoryFlags) or/and update domain cache from sdkdom at the end of operation (vzDomainSuspend). There is a known issue however. Frankly speaking it was introduced by ealier patch '[PATCH 6/9] vz: cleanup loading domain code' from a different series. The patch significantly reduced amount of time when the driver lock is held when creating domain from API call or as a result of domain added event from vz sdk. The problem is these two paths race on using thread shared sdkdom as we don't have libvirt domain object and can not lock on it. However this don't invalidates the patch as we can't use the former approach of preadding domain into the list as we need name at least and name is not given by event. Anyway i'm against adding half baked object into the list. Eventually this race can be fixed by extra measures. As to current situation races with different configurations are unlikely and race when adding domain thru vz driver and simultaneous event from vz sdk is not dangerous as configuration is the same. The last group [3] is API calls that need only sdkdom object to make vz sdk call and don't change thread shared sdkdom object or domain cache in any way. For now these are mostly domain snapshot API calls. The changes are similar to those of group 2 - they add extra reference and drop/reacquire the lock on waiting vz async call result. One can simply take the immutable sdkdom object from the cache and drop the lock for the rest of operations but the chosen approach makes implementation of these API calls somewhat similar to those of from group 2 and thus a bit futureproof. As calls of group 3 don't need vz driver domain/vz sdk cache in any way, they are consistent with respect to API calls from groups 1 and 3. There is another exception. Calls to make-snapshot/revert-to-snapshot/migrate are moved to group 2. That is they are made mutually exclusive. The reason is that libvirt API supports control/query only for one job per domain and these are jobs that are likely to be queried/aborted. Appendix. [1] API calls that only query domain cache. (marked [*] are included for a different reason) .domainLookupByID = vzDomainLookupByID, /* 0.10.0 */ .domainLookupByUUID = vzDomainLookupByUUID, /* 0.10.0 */ .domainLookupByName = vzDomainLookupByName, /* 0.10.0 */ .domainGetOSType = vzDomainGetOSType, /* 0.10.0 */ .domainGetInfo = vzDomainGetInfo, /* 0.10.0 */ .domainGetState = vzDomainGetState, /* 0.10.0 */ .domainGetXMLDesc = vzDomainGetXMLDesc, /* 0.10.0 */ .domainIsPersistent = vzDomainIsPersistent, /* 0.10.0 */ .domainGetAutostart = vzDomainGetAutostart, /* 0.10.0 */ .domainGetVcpus = vzDomainGetVcpus, /* 1.2.6 */ .domainIsActive = vzDomainIsActive, /* 1.2.10 */ .domainIsUpdated = vzDomainIsUpdated, /* 1.2.21 */ .domainGetVcpusFlags = vzDomainGetVcpusFlags, /* 1.2.21 */ .domainGetMaxVcpus = vzDomainGetMaxVcpus, /* 1.2.21 */ .domainHasManagedSaveImage = vzDomainHasManagedSaveImage, /* 1.2.13 */ .domainGetMaxMemory = vzDomainGetMaxMemory, /* 1.2.15 */ .domainBlockStats = vzDomainBlockStats, /* 1.2.17 */ .domainBlockStatsFlags = vzDomainBlockStatsFlags, /* 1.2.17 */ .domainInterfaceStats = vzDomainInterfaceStats, /* 1.2.17 */ [*] .domainMemoryStats = vzDomainMemoryStats, /* 1.2.17 */ .domainMigrateBegin3Params = vzDomainMigrateBegin3Params, /* 1.3.5 */ .domainMigrateConfirm3Params = vzDomainMigrateConfirm3Params, /* 1.3.5 */ [2] API calls that use thread shared sdkdom object (marked [*] are included for a different reason) .domainSuspend = vzDomainSuspend, /* 0.10.0 */ .domainResume = vzDomainResume, /* 0.10.0 */ .domainDestroy = vzDomainDestroy, /* 0.10.0 */ .domainShutdown = vzDomainShutdown, /* 0.10.0 */ .domainCreate = vzDomainCreate, /* 0.10.0 */ .domainCreateWithFlags = vzDomainCreateWithFlags, /* 1.2.10 */ .domainReboot = vzDomainReboot, /* 1.3.0 */ .domainDefineXML = vzDomainDefineXML, /* 0.10.0 */ .domainDefineXMLFlags = vzDomainDefineXMLFlags, /* 1.2.12 */ (update part) .domainUndefine = vzDomainUndefine, /* 1.2.10 */ .domainAttachDevice = vzDomainAttachDevice, /* 1.2.15 */ .domainAttachDeviceFlags = vzDomainAttachDeviceFlags, /* 1.2.15 */ .domainDetachDevice = vzDomainDetachDevice, /* 1.2.15 */ .domainDetachDeviceFlags = vzDomainDetachDeviceFlags, /* 1.2.15 */ .domainSetUserPassword = vzDomainSetUserPassword, /* 1.3.6 */ .domainManagedSave = vzDomainManagedSave, /* 1.2.14 */ .domainSetMemoryFlags = vzDomainSetMemoryFlags, /* 1.3.4 */ .domainSetMemory = vzDomainSetMemory, /* 1.3.4 */ .domainRevertToSnapshot = vzDomainRevertToSnapshot, /* 1.3.5 */ [*] .domainSnapshotCreateXML = vzDomainSnapshotCreateXML, /* 1.3.5 */ [*] .domainMigratePerform3Params = vzDomainMigratePerform3Params, /* 1.3.5 */ [*] .domainUpdateDeviceFlags = vzDomainUpdateDeviceFlags, /* 2.0.0 */ prlsdkHandleVmConfigEvent [3] API calls that do not use thread shared sdkdom object .domainManagedSaveRemove = vzDomainManagedSaveRemove, /* 1.2.14 */ .domainSnapshotNum = vzDomainSnapshotNum, /* 1.3.5 */ .domainSnapshotListNames = vzDomainSnapshotListNames, /* 1.3.5 */ .domainListAllSnapshots = vzDomainListAllSnapshots, /* 1.3.5 */ .domainSnapshotGetXMLDesc = vzDomainSnapshotGetXMLDesc, /* 1.3.5 */ .domainSnapshotNumChildren = vzDomainSnapshotNumChildren, /* 1.3.5 */ .domainSnapshotListChildrenNames = vzDomainSnapshotListChildrenNames, /* 1.3.5 */ .domainSnapshotListAllChildren = vzDomainSnapshotListAllChildren, /* 1.3.5 */ .domainSnapshotLookupByName = vzDomainSnapshotLookupByName, /* 1.3.5 */ .domainHasCurrentSnapshot = vzDomainHasCurrentSnapshot, /* 1.3.5 */ .domainSnapshotGetParent = vzDomainSnapshotGetParent, /* 1.3.5 */ .domainSnapshotCurrent = vzDomainSnapshotCurrent, /* 1.3.5 */ .domainSnapshotIsCurrent = vzDomainSnapshotIsCurrent, /* 1.3.5 */ .domainSnapshotHasMetadata = vzDomainSnapshotHasMetadata, /* 1.3.5 */ .domainSnapshotDelete = vzDomainSnapshotDelete, /* 1.3.5 */ [4] Known issues. 1. accidental errors on getting network statistics 2. race with simultaneous use of thread shared domain object on paths of adding domain thru API and adding domain on vz sdk domain added event. 2016-07-20 Nikolay Shirokovskiy vz: keep naming convention for domain objects see 4385b868 vz: remove redundant variable in prlsdkHandleVmAddedEvent vz: use state variable sdkdom in prlsdkApplyConfig sdk domain handle is unique per connection so there is no sense to query it again if we have it in vzDomObjPtr. Side effect of prlsdkSdkDomainLookupByUUID is refreshing domain config is of no use too as PrlVm_BeginEdit do it too. 2016-07-20 John Ferlan storage: Add extra failure condition for luks volume creation Commit id '5e46d7d6' did not take into account that usage of a luks volume will require usage of the master key encrypted passphrase for a QEMU environment. So rather than allow creation of something that won't be usable, just fail the creation. 2016-07-20 John Ferlan qemu: Disallow usage of luks encryption if aes secret not possible Resolves a CI test integration failure with a RHEL6/Centos6 environment. In order to use a LUKS encrypted device, the design decision was to generate an encrypted secret based on the master key. However, commit id 'da86c6c' missed checking for that specifically. When qemuDomainSecretSetup was implemented, a design decision was made to "fall back" to a plain text secret setup if the specific cipher was not available (e.g. virCryptoHaveCipher(VIR_CRYPTO_CIPHER_AES256CBC)) as well as the QEMU_CAPS_OBJECT_SECRET. For the luks encryption setup there is no fall back to the plaintext secret, thus if that gets set up by qemuDomainSecretSetup, then we need to fail. Also, while the qemuxml2argvtest has set the QEMU_CAPS_OBJECT_SECRET bit, it didn't take into account the second requirement that the ability to generate the encrypted secret is possible. So modify the test to not attempt to run the luks-disk if we know we don't have the encryption algorithm. 2016-07-20 John Ferlan storage: Fix error path virStorageBackendCreateQemuImgCheckEncryption didn't return -1 if there were no secrets. qemu: Move setting of encobjAdded for qemuDomainAttachSCSIDisk A post push realization that the boolean should be set inside the condition qemu: Move setting of obj bools for qemuDomainAttachVirtioDiskDevice A post push realization that the setting of the boolean needed to be inside the if condition. 2016-07-19 Ján Tomko hvsupport: skip non-matching lines early Most of the lines we look at are not going to match one of the driver types contained in $groups_regex. Move on to the next line if it does not contain any of them early. This speeds up the script execution by 50%, since this simple regex does not have any capture groups. 2016-07-19 Ján Tomko hvsupport: construct the group regex upfront The %groups hash contains all the driver types (e.g. virHypervisorDriver or virSecretDriver). When searching for all the APIs that are implemented by a driver of that specific driver type, we keep iterating over the %groups hash on every line we look at, then matching against the driver type. This is inefficient because it prevents perl from caching the regex and it executes the regex once for every driver type, even though one regex matching excludes all the others, since all the driver types are different. Construct the regex containing all the driver types upfront to save about 6.4s (~98%) of the script execution time. 2016-07-19 Ján Tomko hvsupport: use a regex instead of XML::XPath When generating the hvsupport.html.in file, we parse the -api.xml files generated by apibuild.py to know in which HTML file the API function is. Doing an XPath query for every single 'function' element in the file is inefficient. Since the XML file is generated by another of our build scripts (apibuild.py, using Python's standard 'output.write' XML library), just find the function name->file mapping by a regex upfront. Also add a note about this next to the line that generates it in apibuild.py and do not check if XML::XPath is installed in bootstrap since we no longer use it. 2016-07-19 Cédric Bosdonnat lxc: errors after the handshake won't be reported Any error happening after the hand shake in the lxc controller will not result in a failure as errors are checked during the handshake. Move the handshake after the last possible error. virt-aa-helper: better write denials handling Better fix replacing c726af2d: introducing an 'R' permission to add read rule, but no explicit deny write rule. 2016-07-19 John Ferlan qemu: Add luks support for domain disk Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1301021 Generate the luks command line using the AES secret key to encrypt the luks secret. A luks secret object will be in addition to a an AES secret. For hotplug, check if the encinfo exists and if so, add the AES secret for the passphrase for the secret object used to decrypt the device. Modify/augment the fakeSecret* in qemuxml2argvtest in order to handle find a uuid or a volume usage with a specific path prefix in the XML (corresponds to the already generated XML tests). Add error message when the 'usageID' is not 'mycluster_myname'. Commit id '1d632c39' altered the error message generation to rely on the errors from the secret_driver (or it's faked replacement). Add the .args output for adding the LUKS disk to the domain 2016-07-19 John Ferlan qemu: Alter the qemuDomainGetSecretAESAlias to add new arg Soon we will be adding luks encryption support. Since a volume could require both a luks secret and a secret to give to the server to use of the device, alter the alias generation to create a slightly different alias so that we don't have two objects with the same alias. qemu: Add secinfo for hotplug virtio disk Commit id 'a1344f70a' added AES secret processing for RBD when starting up a guest. As such, when the hotplug code calls qemuDomainSecretDiskPrepare an AES secret could be added to the disk about to be hotplugged. If an AES secret was added, then the hotplug code would need to generate the secret object because qemuBuildDriveStr would add the "password-secret=" to the returned 'driveStr' rather than the base64 encoded password. 2016-07-19 John Ferlan storage: Add support to create a luks volume Partially resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1301021 If the volume xml was looking to create a luks volume take the necessary steps in order to make that happen. The processing will be: 1. create a temporary file (virStorageBackendCreateQemuImgSecretPath) 1a. use the storage driver state dir path that uses the pool and volume name as a base. 2. create a secret object (virStorageBackendCreateQemuImgSecretObject) 2a. use an alias combinding the volume name and "_luks0" 2b. add the file to the object 3. create/add luks options to the commandline (virQEMUBuildLuksOpts) 3a. at the very least a "key-secret=%s" using the secret object alias 3b. if found in the XML the various "cipher" and "ivgen" options 2016-07-19 John Ferlan vsh: Properly initialize res The 'res' variable was only being initialized to NULL in the if (!state) path; however, that path never used res and evenutally res is assigned one of two results based on a pair of if then else if conditions. If for some reason neither of those paths was taken and the (!state) path wasn't taken, then 'res' would be indeterminate. Found by Coverity, probably a false positive based on code paths, but better safe than sorry for the future. 2016-07-19 John Ferlan conf: Need to check for glisten before accessing When formatting the graphics data for TYPE_SPICE, check if the glisten is NULL before blindly referencing Found by Coverity 2016-07-19 John Ferlan tests: Need to check return of virGetLastError Cannot assume virGetLastError returns non-NULL value - modify the code to fetch err and check if err && err->code Found by Coverity 2016-07-19 John Ferlan tools: Fix comparison in virLoginShellGetShellArgv Commit id '740e4d70' altered the logic to fetch the sysconf values and added a new virConfGetValueStringList which returns -1 on failure, 0 if missing, and 1 if the value was present. However, the caller only checked !shargv which caught Coverity's attention since the following VIR_ALLOC_N(*shargv, 2) would be a NULL ptr deref 2016-07-19 John Ferlan util: Fix incorrect VIR_FREE in virConfGetValueStringList Since we VIR_ALLOC_N to *values, the VIR_FREE should be done likewise 2016-07-19 Erik Skultety virt-admin: Output srv-threadpool-info data as unsigned int rather than signed Internally, all the data are represented as unsigned int, it is also documented in the header file that users should use our exported constants that also indicate that the data should be unsigned int. However, when polling for the current server threadpool's configuration, virt-admin uses an incorrect formatting parameter '%d' for printf. Instead, virt-admin should use formatting parameter '%u'. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1356769 2016-07-19 John Ferlan qemu: Move and rename qemuBufferEscapeComma Move to virqemu and rename to virQEMUBuildBufferEscapeComma. This can then be shared by the storage driver when it builds the command line for qemuimg qemu: Alter error path cleanup for qemuDomainAttachMemory A recent adjustment to qemuDomainAttachRNGDevice to properly cleanup the props object after a qemuMonitorAddObject also would affect this code. Alter the cleanup to be similar to RNG changes. qemu: Alter error path cleanup for qemuDomainAttachRNGDevice Based on recent review comment - rather than have a spate of goto failxxxx, change to a boolean based model. Ensures that the original error can be preserved and cleanup is a bit more orderly if more objects are added. qemu: Alter error path cleanup for qemuDomainAttachChrDevice Based on recent review comment - rather than have a spate of goto failxxxx, change to a boolean based model. Ensures that the original error can be preserved and cleanup is a bit more orderly if more objects are added. qemu: Alter error path cleanup for qemuDomainAttachSCSIDisk Based on recent review comment - rather than have a spate of goto failxxxx, change to a boolean based model. Ensures that the original error can be preserved and cleanup is a bit more orderly if more objects are added. qemu: Alter error path cleanup for qemuDomainAttachVirtioDiskDevice Based on recent review comment - rather than have a spate of goto failxxxx, change to a boolean based model. Ensures that the original error can be preserved and cleanup is a bit more orderly if more objects are added. qemu: Alter error path cleanup for qemuDomainAttachHostSCSIDevice Based on recent review comment - rather than have a spate of goto failxxxx, change to a boolean based model. Ensures that the original error can be preserved and cleanup is a bit more orderly if more objects are added. 2016-07-19 Erik Skultety fs: Fix '<' comparison of value produced by logical not '!' Commit da665fbd introduced the following condition to virLXCProcessEnsureRootFS and openvzReadFSConf: if (!( = virDomainFSDefNew()) < 0) which broke the build on fedora with GCC 5.3.1: "logical not is only applied to the left hand side of comparison". 2016-07-19 Julio Faracco security: compilation error due to wrong parameter for vah_add_path(). The commit da665fbd introduced virStorageSourcePtr inside the structure _virDomainFSDef. This is causing an error when libvirt is being compiled. make[3]: Entering directory `/media/julio/8d65c59c-6ade-4740-9cdc-38016a4cb8ae /home/julio/Desktop/virt/libvirt/src' CC security/virt_aa_helper-virt-aa-helper.o security/virt-aa-helper.c: In function 'get_files': security/virt-aa-helper.c:1087:13: error: passing argument 2 of 'vah_add_path' from incompatible pointer type [-Werror] if (vah_add_path(&buf, fs->src, "rw", true) != 0) ^ security/virt-aa-helper.c:732:1: note: expected 'const char *' but argument is of type 'virStorageSourcePtr' vah_add_path(virBufferPtr buf, const char *path, const char *perms, bool recursive) ^ cc1: all warnings being treated as errors Adding the attribute "path" from virStorageSourcePtr fixes this issue. 2016-07-19 Nikolay Shirokovskiy vz: add tcp and udp serial device support vz supports only a subset of tcp and udp parameters. 1. tcp type supports only 'raw' protocol. 2. udp type supports only same parameters of 'host' and 'service' for 'bind' and 'connect'. 2016-07-19 Nikolay Shirokovskiy vz: add mode of unix socket serial device to xml dump vz: don't fail unregister on sending event error vz: use single variable for domain vz: dont remove domain from list on client object error After domain is in the domains list let's keep it there. This is approach taken by qemu driver and vz vzDomainMigrateFinish3Params too. It quite reasonable, driver domain object is fully constructed and can be discovered by client later. 2016-07-19 Nikolay Shirokovskiy vz: cleanup loading domain code 9c14a9ab introduced vzNewDomain function to enlist libvirt domain object before actually creating vz sdk domain. Fix should fix race on same vz sdk domain added event where libvirt domain object is enlisted too. But later eb5e9c1e added locked checks for adding livirtd domain object to list on vz sdk domain added event. Thus now approach of 9c14a9ab is unnecessary complicated. See we have otherwise unuseful prlsdkGetDomainIds function only to create minimal domain definition to create libvirt domain object. Also vzNewDomain is difficult to use as it creates partially constructed domain object. Let's move back to original approach where prlsdkLoadDomain do all the necessary job. Another benefit is that we can now take driver lock for bare minimum and in single place. Reducing locking time have small disadvatage of double parsing on race conditions which is typical if domain is added thru vz driver. Well we have this double parse inevitably with current vz sdk api on any domain updates so i would not take it here seriously. Performance events subscribtion is done before locked check and therefore could be done twice on races but this is not the problem. 2016-07-19 Nikolay Shirokovskiy vz: use domain list infrastructure to deal with private domain vz: make error handling idiomatic in prlsdkCreateVm vz: fix leaks in prlsdkCreate* functions vz: restore accidentally removed locks around close callback calls vz: remove unnecessary labels in simple API calls 2016-07-18 Olga Krishtal vz: support filesystem type volume Vz containers are able to use ploop volumes from storage pools to work upon. To use filesystem type volume, pool name and volume name should be specifaed in : The information about pool and volume is stored in ct dom configuration: libvirt://localhost/pool_name/vol_name and can be easily obtained via PrlVmDevHd_GetStorageURL sdk call. The only shorcoming: if storage pool is moved somewhere the ct should be redefined in order to refresh the information aboot path to root.hdd 2016-07-18 Olga Krishtal vz: refactoring of prlsdkCreateCt We do not need to check domainf fs type there, because it is done in prlsdkCheckUnsupportedParams. 2016-07-18 Olga Krishtal devices: filesystems: added volume type New type of is introduced. This patch allows to use volumes for storing the filesystem, that is accessed from the guest e.g. root directory for container. To take advantage of volumes as a backend of filesystem volume and pool names should be specified: 2016-07-18 Olga Krishtal filesystem: adds possibility to use storage pool as fs source vz: fixed null-pointer dereference in applying graphic params 2016-07-18 Nikolay Shirokovskiy vz: fix destination domain synchronization Adding domain to domain list on preparation step is not correct. First domain is not fully constructed - domain definition is missing. Second we can't use VIR_MIGRATE_PARAM_DEST_XML parameter to parse definition as vz sdk can patch it by itself. Let's add/remove domain on finish step. This is for synchronization purpose only so domain is present/absent on destination after migration completion. Actually domain object will probably be created right after actual vz sdk migration start by vz sdk domain defined event. We can not and should not sync domain cache on error path in finish step of migration. We can not as we really don't know what is the reason of cancelling and we should not as user should not make assumptions on state on error path. What we should do is cleaning up temporary migration state that is induced on prepare step but we don't have one. Thus cancellation should be noop. 2016-07-18 Nikolay Shirokovskiy vz: fix memory leaks in prlsdkLoadDomains Free sdkdom on any result of prlsdkNewDomainByHandle. vz: fix missed defined domain event libvirt domain defined event is issued only on correspondent vz sdk event. But in case event delivered before domain is added to domain list we can mistakenly skip this event if prlsdkNewDomainByHandle return NULL in case of domain is discovered in the list under the driver lock. Let's return domain object in this case. Now prlsdkNewDomainByHandle returns NULL only in case of error which is more convinient. vz: don't pass empty and unused fields in migration cookie The first version of migration cookie was rather dumb resulting in passing empty or unused fields here and there. Add flags to specify what to bake to and eat from cookie so we deal only with meaningful data. However for backwards compatibility we still need to pass at least some faked fields sometimes. 2016-07-18 Jiri Denemark qemu: Fix migration from old libvirt Older libvirt versions send persistent XML in a migration cookie even when VIR_MIGRATE_PERSIST_DEST flag is not used, but current libvirt properly fails if the cookie contains unexpected flags. Thus migration from old libvirt fails with internal error: Unsupported migration cookie feature persistent unless VIR_MIGRATE_PERSIST_DEST flag is set. https://bugzilla.redhat.com/show_bug.cgi?id=1320500 2016-07-18 Cole Robinson test: Add scsi vport nodedev to test:///default A nodedev device definition like this is required for testing NodeDeviceCreateXML and NodeDeviceDestroy. So unless it's part of the stock test:///default set there's no way to actually invoke those functions for the default URI test: Have test:///default open use file parsing helpers Convert the individual XML documents into one big XML document in the format expected by the non-default test://$PATH URI, and use the same internal helpers for assembling the driver contents. test: Move testOpenDefault definition later Upcoming patches need this defined later test: Introduce testOpenParse 2016-07-18 Andrea Bolognani util: conf: Rename VIR_CONF_{U,}LONG -> VIR_CONF_{U,}LLONG Since commit 6381c89f8cce, we're storing long long integers instead of long integers. Rename the corresponding virConfType value accordingly. util: conf: Clarify choice between VIR_CONF_LONG and VIR_CONF_ULONG We use unsigned long long integers unless we need to store a negative value. Rewrite the condition to make this more obvious. util: conf: Fix parameters alignment The parameters for virConfGetValueLLong() were not aligned properly. util: conf: Fix comment for virConfGetValueULLong() The name of the function is not virConfGetValueULongLong(). util: conf: Claim the proper range for signed numbers virConfGetValueLLong() errors out if the value is too big to fit into a long long integer, but claims the supported range to be (0,LLONG_MAX) instead of (LLONG_MIN,LLONG_MAX). util: conf: Add integer casts For good measure. util: conf: Improve virConfGet*() logic When parsing numeric values, we always store them as unsigned unless they're negative. We can use this fact to simplify the logic by removing a bunch of unnecessary checks. util: conf: Use long long when parsing Commit 6381c89f8cce changed virConfValue to store long long integers instead of long integers; however, the temporary variable used in virConfParseLong() was not updated accordingly, causing trouble for 32-bit machines. 2016-07-18 Ján Tomko hvsupport: Introduce parseSymsFile The code for parsing the different public syms files only differs in the filenames and version prefix. Unify it to a single subroutine. 2016-07-18 Ján Tomko Store USB port path as an array of integers In preparation to tracking which USB addresses are occupied. Introduce two helper functions for printing the port path as a string and appending it to a virBuffer. 2016-07-18 Ján Tomko Allow omitting USB port We were requiring a USB port path in the schema, but not enforcing it. Omitting the USB port would lead to libvirt formatting it as (null). Such domain cannot be started and will disappear after libvirtd restart (since it cannot parse back the XML). Only format the port if it has been specified and mark it as optional in the XML schema. 2016-07-18 Jiri Denemark qemu: Drop default channel path during migration Migration to an older libvirt (pre v1.3.0-175-g7140807) is broken because older versions of libvirt generated different channel paths and they didn't drop the default paths when parsing domain XMLs. We'd get such a nice error message: internal error: process exited while connecting to monitor: 2016-07-08T15:28:02.665706Z qemu-kvm: -chardev socket, id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/ domain-3-nest/org.qemu.guest_agent.0,server,nowait: Failed to bind socket to /var/lib/libvirt/qemu/channel/target/domain-3-nest/ org.qemu.guest_agent.0: No such file or directory That said, we should not even format the default paths when generating a migratable XML. https://bugzilla.redhat.com/show_bug.cgi?id=1320470 2016-07-18 Jiri Denemark qemu: Copy complete domain def in qemuDomainDefFormatBuf Playing directly with our live definition, updating it, and reverting it back once we are done is very nice and it's quite dangerous too. Let's just make a copy of the domain definition if needed and do all tricks on the copy. https://bugzilla.redhat.com/show_bug.cgi?id=1320470 2016-07-15 Daniel P. Berrange virconf: skip some range checks if SSIZE_MAX >= LLONG_MAX If size_t is the same size as long long, then we can skip some of the range checks. This avoids triggering some bogus compiler warning messages. virconf: clarify type range checks for integers The virConf 'l' field is a 'signed long long', so whenever the 'type' field is VIR_CONF_ULONG, we should explicitly cast 'l' to a 'unsigned long long' before doing range checks. 2016-07-15 Michal Privoznik virConfGetValueSSizeT: Fix build on 32 bits This function tries to get a ssize_t value from a config file. But before returning it, it checks whether the value would fit in ssize_t and if not an error is printed out among with the range for the ssize_t type. However, on some platforms SSIZE_MAX may actually be a signed long type: util/virconf.c: In function 'virConfGetValueSSizeT': util/virconf.c:1268:9: error: format '%zd' expects argument of type 'signed size_t', but argument 9 has type 'long int' [-Werror=format=] virReportError(VIR_ERR_INTERNAL_ERROR, ^ $ grep -r SSIZE_MAX /usr/include/ /usr/include/bits/posix1_lim.h:#ifndef SSIZE_MAX /usr/include/bits/posix1_lim.h:# define SSIZE_MAX LONG_MAX 2016-07-14 John Ferlan conf: Revert changes to add new secret type "passphrase" Revert the remainder of commit id 'c84380106' docs: Update docs to reflect LUKS secret changes Commit id's 'c8438010', '9bbf0d7e', and '2552fec24' altered the documentation to describe adding a 'passphrase' type secret usage model in order to reference the secret for a luks volume. After commit, it was deemed that a 'volume' usage model should be used, so adjust the various documents in order rephrase descriptions in order to follow the correct usage model. 2016-07-14 John Ferlan tests: Adjust LUKS tests to use 'volume' secret type Commit id's '9bbf0d7e6' and '2552fec24' added some XML parsing tests for a LUKS volume to use a 'passphrase' secret format. After commit, this was deemed to be incorrect, so covert the various tests to use the volume usage format where the 'usage' is the path to the volume rather than a user defined name string. Also, removed the qemuxml2argv-luks-disk-cipher.xml since it was just a duplicate of qemuxml2argv-luks-disks.xml. 2016-07-14 Andrea Bolognani tests: command: Fix build on ppc64/aarch64 Commit ca10bb040fcf introduced a new test that fails to build on at least some architectures: commandtest.c: In function 'test25': commandtest.c:1121:5: error: comparison is always true due to limited range of data type [-Werror=type-limits] if (rv >= 0) { ^ Change the type of 'rv' from char to int, which is the proper return type for virCommandExec() anyway. 2016-07-13 Andrea Bolognani spec: Fix indentation Commit ffc49e579c14 broke syntax-check: cppi: libvirt.spec.in: line 622: not properly indented cppi: libvirt.spec.in: line 624: not properly indented cppi: libvirt.spec.in: line 640: not properly indented cppi: libvirt.spec.in: line 642: not properly indented maint.mk: incorrect preprocessor indentation cfg.mk:697: recipe for target 'sc_spec_indentation' failed Indent the new conditionals properly. 2016-07-13 Martin Kletzander conf: Make really sure we don't access non-existing vCPUs again MinGW complained that we might be dereferencing a NULL pointer. While that can't be true, the logic certainly allows for that. ../../src/conf/domain_conf.c: In function 'virDomainDefPostParse': ../../src/conf/domain_conf.c:4224:18: error: potential null pointer dereference [-Werror=null-dereference] if (!vcpu->online && vcpu->cpumask) { ~~~~^~~~~~~~ 2016-07-13 Daniel P. Berrange libvirt.spec.in: require systemd-container on >= f24 The systemd-machined tools libvirt uses were split into a systemd-container RPM. Without depending on this, libvirt may silently fallback to the non-systemd cgroup impl which is not desirable. 2016-07-13 Andrea Bolognani qemu: command: Fix awkward formatting 2016-07-13 Martin Kletzander conf: Make really sure we don't access non-existing vCPUs MinGW complained that we might be dereferencing a NULL pointer. While that can't be true, the logic certainly allows for that. src/conf/domain_conf.c: In function 'virDomainDefGetVcpuPinInfoHelper': src/conf/domain_conf.c:1545:17: error: potential null pointer dereference [-Werror=null-dereference] if (vcpu->cpumask) ~~~~^~~~~~~~~ 2016-07-13 Maxim Perevedentsev dnsmasq: disable IPv6 default gateway in RA for isolated networks IPv6 RA always contains an implicit default route via the link-local address of the source of RA. This forces the guest to install a route via isolated network, which may disturb the guest's networking in case of multiple interfaces. More info in 013427e6e733f7a662f4e8a9c11f7dad4cd65e3f. The validity of this route is controlled by "default [route] lifetime" field of RA. If the lifetime is set to 0 seconds, then no route is installed by receiver. dnsmasq 2.67+ supports "ra-param=,," option. We pass "ra-param=*,0,0" (here, RA_interval=0 means default) to disable default gateway in RA for isolated networks. 2016-07-13 Maxim Perevedentsev Fix message about dnsmasq BINDTODEVICE capability. 2016-07-13 Yan Fu qemu: getAutoDumpPath() return value should be dumpfile not domname. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1354238 So we spend some time and effort constructing perfect file name for an automatic coredump of a domain, but then just leak it and use the domain name anyway. This is probably due to a silly mistake that slipped even through review. 2016-07-12 Jim Fehlig systemd: fix ready notification on abstract socket At least with systemd v210, NOTIFY_SOCKET is abstact, e.g. @/org/freedesktop/systemd1/notify. sendmsg() fails on such a socket with "Connection refused". The unix(7) man page contains the following details wrt abstract socket addresses abstract: an abstract socket address is distinguished (from a pathname socket) by the fact that sun_path[0] is a null byte ('\0'). The socket's address in this namespace is given by the additional bytes in sun_path that are covered by the specified length of the address structure. (Null bytes in the name have no special significance.) So we need to be more precise about the address length, setting it to the sizeof sa_family_t + length of address copied to sun_path instead of setting it to the sizeof the entire sockaddr_un struct. Resolves: https://bugzilla.opensuse.org/show_bug.cgi?id=987668 2016-07-12 Tomáš Golembiovský esx: Fetch snapshot info directly for filtering When fetching domains with virConnectListAllDomains() and when filtering by snapshot existence is requested the ESX driver first lists all the domains and then check one-by-one for snapshot existence. This process takes unnecessarily long time. To significantly improve the time necessary to finish the query we can request the snapshot related info directly when querying the list of domains from VMware. 2016-07-12 Eric Blake build: drop hack for old mingw ssize_t The old 32-bit-only mingw project had a mismatch in its that omitted ssize_t, but where size_t was a different rank than int as picked by gnulib. But now that mingw64 (both 32- and 64-bit) is the more popular platform (Fedora has dropped mingw in favor of mingw64), we no longer need to carry a hack to gnulib to cater to the old mingw. This reverts part of commit 1012dc29333. build: virrandommock.c not needed on mingw We can't mock tests on Mingw, which lacks dlopen() and friends; follow the paradigms used in other mock files of conditionally compiling nothing when not building for Linux. 2016-07-12 Michal Privoznik virCommandExec: Report error if execve fails In an unlikely event of execve() failing, the virCommandExec() function does not report any error, even though checks that are at the beginning of the function are verbose when failing. 2016-07-12 Olga Krishtal storage: dir: adapts .wipeVol for ploop volumes The modification of .volWipe callback wipes ploop volume using one of given wiping algorithm: dod, nnsa, etc. However, in case of ploop volume we need to reinitialize root.hds and DiskDescriptor.xml. v2: - added check on ploop tools presens - virCommandAddArgFormat changed to virCommandAddArg 2016-07-12 Ján Tomko qemu: format intel-iommu on the command line results in: -device intel-iommu https://bugzilla.redhat.com/show_bug.cgi?id=1235580 2016-07-12 Ján Tomko Add QEMU_CAPS_DEVICE_INTEL_IOMMU Check whether QEMU supports -device intel-iommu Note that the presence of this option does not mean that it's usable because of a bug in earlier QEMU versions, but it's better than nothing. https://bugzilla.redhat.com/show_bug.cgi?id=1235580 2016-07-12 Ján Tomko Introduce device A device with an attribute 'model', with just one model so far: ... https://bugzilla.redhat.com/show_bug.cgi?id=1235580 2016-07-12 Ján Tomko test-wrap-argv: set cutoff at 78 characters For every but the last argument, we also need space for a space and a backslash. Rewrap everything longer than 78 characters. 2016-07-12 Ján Tomko test-wrap-argv: print diff instead of the incorrectly wrapped file Commit c9c03ea stopped creating an intermediate file during syntax-check to save on execution time. It also switched to outputting the whole incorrectly wrapped file instead of a diff needed to fix it. Feed the newly wrapped file to diff via a pipe. Note that fixing it by running test-wrap-argv.pl --in-place or the unit test with VIR_TEST_REGENERATE_OUTPUT is easier. 2016-07-12 Ján Tomko testutils: only rewrap args files test-wrap-argv.pl does not know how to rewrap other files. 2016-07-12 Ján Tomko testutils: find perl early Commit 843a70a changed test-wrap-argv.pl to use /usr/bin/env perl instead of /usr/bin/perl However when called from qemuxml2argvtest with VIR_TEST_REGENERATE_OUTPUT, PATH is set to '/bin'. Find the path to perl early in virTestMain, in case we are going to need it later after we've overridden PATH. 2016-07-12 Ján Tomko qemuxml2argvtest: drop empty pseries-vio-address-clash.args Unused since its introduction in commit 4b942fe. 2016-07-12 Luyao Huang tests: add missing directories in EXTRA_DIST In commit ec5dcf2a and b0b4a35c we have moved qemuhotplugtest's XMLs to new directories but forgot to fix the Makefile. Add 2 directories in EXTRA_DIST to fix broken VPATH build. Also remove now unused qemuhotplugtestdata directory from the Makefile as well as from the tree. 2016-07-12 Daniel P. Berrange remote: convert to typesafe virConf accessors sanlock: convert to typesafe virConf accessors lockd: convert to typesafe virConf accessors selinux: convert to typesafe virConf accessors virt-login-shell: convert to typesafe virConf accessors uri: convert to typesafe virConf accessors libxl: convert to typesafe virConf accessors lxc: convert to typesafe virConf accessors virtlogd: convert to typedef virConf accessors virtlockd: convert to typesafe virConf accessors libvirt: convert to typesafe virConf accessors qemu: convert to typesafe virConf accessors libvirtd: convert to typesafe virConf accessors The libvirtdconftest was previously used to test data type handling of the libvirtd config file. Now we're using the typedef APIs, this test case has little value, and is pretty hard to fixup with deal with the new APIs. virconf: add typed value accessor methods Currently many users of virConf APIs are defining the same macros for calling virConfValue() and then doing type checking. To remove this repeated code, add a set of typesafe accessor methods. virconf: fix off-by-1 when appending \n to config file If the config file does not end with a \n, the parser will append one. When re-allocating the array though, it is mistakenly assuming that 'len' is the length including the trailing NUL, but it does not. So we must add 2 to len, when reallocating, not 1. 2016-07-12 Daniel P. Berrange tests: remove pointless virconftest.sh wrapper The virconftest is different from all our other tests in that the C program only tests a single in/out config file pair. It relies on a shell wrapper to invoke it once for each test file. This gets rid of the shell wrapper and makes the C program actually run over each test file using the normal test pattern. 2016-07-11 Tomasz Flendrich qemuhotplugtest: Move domain and device XMLs to different directories This way we can safely differentiate what XMLs contain whole domain definitions and which contain just devices. Thanks to that we can test the domain XMLs in virschematest again. qemuhotplugtest: Move all XMLs to one directory This makes the search for related XMLs easier, plus they are not used in the xml2argv tests anyway. This also makes future patches cleaner. While on that remove unnecessary '-hotplug' from the filenames. 2016-07-11 Michal Privoznik qemuxml2argvmock: Don't leak @netdef->ifname In the mock, we have a stub for virNetDevTapCreate(). However, the mocked version does not exactly as it's native counterpart. The function receives a string, which is an interface name that caller would like to have, but it's not guaranteed that they will get just that one. If they don't, the function free()-s the one passed and returns the new one. Just like the mocked version. But what is the mocked version missing is the free(). ==1068== 6 bytes in 1 blocks are definitely lost in loss record 9 of 132 ==1068== at 0x4C29F80: malloc (vg_replace_malloc.c:296) ==1068== by 0xDE13356: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4) ==1068== by 0xAE2333E: virXMLPropString (virxml.c:479) ==1068== by 0xAE45975: virDomainNetDefParseXML (domain_conf.c:9038) ==1068== by 0xAE5C0BB: virDomainDefParseXML (domain_conf.c:16734) ==1068== by 0xAE5EB96: virDomainDefParseNode (domain_conf.c:17444) ==1068== by 0xAE5EA05: virDomainDefParse (domain_conf.c:17391) ==1068== by 0xAE5EA93: virDomainDefParseFile (domain_conf.c:17415) ==1068== by 0x433430: testCompareXMLToArgvFiles (qemuxml2argvtest.c:278) ==1068== by 0x433A18: testCompareXMLToArgvHelper (qemuxml2argvtest.c:414) ==1068== by 0x446ED4: virTestRun (testutils.c:179) ==1068== by 0x43A099: mymain (qemuxml2argvtest.c:1016) 2016-07-11 Michal Privoznik qemuxml2argvtest: Don't leak dummy monitor It's just test, but why leak it? ==26971== 20 bytes in 1 blocks are definitely lost in loss record 623 of 704 ==26971== at 0x4C29F80: malloc (vg_replace_malloc.c:296) ==26971== by 0xE560447: vasprintf (vasprintf.c:76) ==26971== by 0xAE0DEE2: virVasprintfInternal (virstring.c:480) ==26971== by 0xAE0DFF7: virAsprintfInternal (virstring.c:501) ==26971== by 0x4751F3: qemuProcessPrepareMonitorChr (qemu_process.c:2651) ==26971== by 0x4334B1: testCompareXMLToArgvFiles (qemuxml2argvtest.c:297) ==26971== by 0x4339AC: testCompareXMLToArgvHelper (qemuxml2argvtest.c:413) ==26971== by 0x446E7A: virTestRun (testutils.c:179) ==26971== by 0x445D33: mymain (qemuxml2argvtest.c:2029) ==26971== by 0x44886F: virTestMain (testutils.c:969) ==26971== by 0x445D9B: main (qemuxml2argvtest.c:2036) 2016-07-11 Michal Privoznik qemuDomainObjPrivateFree: Free @masterKey too This one's a bit more complicated. In qemuProcessPrepareDomain() a master key for encrypting secret for ciphered disks is created. This object lives within qemuDomainObjPrivate object. It is freed in qemuProcessStop(), but if nobody calls it (for instance like our qemuxml2argvtest does), the key object leaks. ==17078== 32 bytes in 1 blocks are definitely lost in loss record 633 of 707 ==17078== at 0x4C2C070: calloc (vg_replace_malloc.c:623) ==17078== by 0xAD924DF: virAllocN (viralloc.c:191) ==17078== by 0x5050BA6: virCryptoGenerateRandom (qemuxml2argvmock.c:166) ==17078== by 0x453DC8: qemuDomainMasterKeyCreate (qemu_domain.c:678) ==17078== by 0x47A36B: qemuProcessPrepareDomain (qemu_process.c:4913) ==17078== by 0x47C728: qemuProcessCreatePretendCmd (qemu_process.c:5542) ==17078== by 0x433698: testCompareXMLToArgvFiles (qemuxml2argvtest.c:332) ==17078== by 0x4339AC: testCompareXMLToArgvHelper (qemuxml2argvtest.c:413) ==17078== by 0x446E7A: virTestRun (testutils.c:179) ==17078== by 0x445BD9: mymain (qemuxml2argvtest.c:2022) ==17078== by 0x44886F: virTestMain (testutils.c:969) ==17078== by 0x445D9B: main (qemuxml2argvtest.c:2036) 2016-07-11 Michal Privoznik qemuBuildCpuCommandLine: Don't leak @buf Just like every other qemuBuild*CommandLine() function, this uses a buffer to hold partial cmd line strings too. However, if there's an error, the control jumps to 'cleanup' label leaving the buffer behind and thus leaking it. ==2013== 1,006 bytes in 1 blocks are definitely lost in loss record 701 of 711 ==2013== at 0x4C29F80: malloc (vg_replace_malloc.c:296) ==2013== by 0x4C2C32F: realloc (vg_replace_malloc.c:692) ==2013== by 0xAD925A8: virReallocN (viralloc.c:245) ==2013== by 0xAD95EA8: virBufferGrow (virbuffer.c:130) ==2013== by 0xAD95F78: virBufferAdd (virbuffer.c:165) ==2013== by 0x5097F5: qemuBuildCpuModelArgStr (qemu_command.c:6339) ==2013== by 0x509CC3: qemuBuildCpuCommandLine (qemu_command.c:6437) ==2013== by 0x51142C: qemuBuildCommandLine (qemu_command.c:9174) ==2013== by 0x47CA3A: qemuProcessCreatePretendCmd (qemu_process.c:5546) ==2013== by 0x433698: testCompareXMLToArgvFiles (qemuxml2argvtest.c:332) ==2013== by 0x4339AC: testCompareXMLToArgvHelper (qemuxml2argvtest.c:413) ==2013== by 0x446E7A: virTestRun (testutils.c:179) 2016-07-11 Michal Privoznik virStorageEncryptionSecretFree: Don't leak secret lookup definition When storage secret is parsed in virStorageEncryptionSecretParse(), virSecretLookupParseSecret() which allocates some memory. This is however never freed. ==21711== 134 bytes in 6 blocks are definitely lost in loss record 70 of 85 ==21711== at 0x4C29F80: malloc (vg_replace_malloc.c:296) ==21711== by 0xBCA0356: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4) ==21711== by 0xA9F432E: virXMLPropString (virxml.c:479) ==21711== by 0xA9D25B0: virSecretLookupParseSecret (virsecret.c:70) ==21711== by 0xA9D616E: virStorageEncryptionSecretParse (virstorageencryption.c:172) ==21711== by 0xA9D66B2: virStorageEncryptionParseXML (virstorageencryption.c:281) ==21711== by 0xA9D68DF: virStorageEncryptionParseNode (virstorageencryption.c:338) ==21711== by 0xAA12575: virDomainDiskDefParseXML (domain_conf.c:7606) ==21711== by 0xAA2CAC6: virDomainDefParseXML (domain_conf.c:16658) ==21711== by 0xAA2FC75: virDomainDefParseNode (domain_conf.c:17472) ==21711== by 0xAA2FAE4: virDomainDefParse (domain_conf.c:17419) ==21711== by 0xAA2FB72: virDomainDefParseFile (domain_conf.c:17443) 2016-07-11 Chen Hanxiao virsh: allow both --uuid and --name at same time #virsh list --uuid --name 49c765a0-25e7-40d0-964f-dac99724b32c c7 918f1dd6-b19f-412b-ba17-d113bad89af8 f23 2016-07-11 Martin Kletzander qemu: Use qemuProcessSetupPid() in qemuProcessSetupVcpu() qemu: Use qemuProcessSetupPid() in qemuProcessSetupEmulator() qemu: Add qemuProcessSetupPid() and use it in qemuProcessSetupIOThread() Setting up cgroups and other things for all kinds of threads (the emulator thread, vCPU threads, I/O threads) was copy-pasted every time new thing was added. Over time each one of those functions changed a bit differently. So create one function that does all that setup and start using it, starting with I/O thread setup. That will shave some duplicated code and maybe fix some bugs as well. 2016-07-11 Daniel P. Berrange Fix logic in qemuDomainObjPrivateXMLParseVcpu The code in qemuDomainObjPrivateXMLParseVcpu for parsing the 'idstr' string was comparing the overall boolean result against 0 which was always true qemu/qemu_domain.c: In function 'qemuDomainObjPrivateXMLParseVcpu': qemu/qemu_domain.c:1482:59: error: comparison of constant '0' with boolean expression is always false [-Werror=bool-compare] if ((idstr && virStrToLong_uip(idstr, NULL, 10, &idx)) < 0 || ^ It was further performing two distinct error checks in the same conditional and reporting a single error message, which was misleading in one of the two cases. This splits the conditional check into two parts with distinct error messages and fixes the logic error. Fixes the bug in commit 5184f398b40a5e0d7d84b86182edcb2b48ab04ba Author: Peter Krempa Date: Fri Jul 1 14:56:14 2016 +0200 qemu: Store vCPU thread ids in vcpu private data objects 2016-07-11 Andrea Bolognani qemu: capabilities: Make virHostCPUGetKVMMaxVCPUs() errors fatal An error in virHostCPUGetKVMMaxVCPUs() means we've been unable to access /dev/kvm, or we're running on a platform that doesn't support KVM in the first place. If that's the case, we shouldn't ignore the error and report domcapabilities even though we know the user won't be able to start any KVM guest. 2016-07-11 Andrea Bolognani util: hostcpu: Drop obsolete compatibility code All Linux releases we support (RHEL6 era) include these definitions. util: hostcpu: Add virHostCPUGetKVMMaxVCPUs() stub If we don't HAVE_LINUX_KVM_H, we can't query /dev/kvm to discover the limits on the number of vCPUs, so we report an error and return a negative value instead. 2016-07-11 Peter Krempa qemu: Store vCPU thread ids in vcpu private data objects Rather than storing them in an external array store them directly. qemu: Add cpu ID to the vCPU pid list in the status XML Note the vcpu ID so that once we allow non-contiguous vCPU topologies it will be possible to pair thread id's with the vcpus. qemu: domain: Extract formating and parsing of vCPU thread ids Further patches will be adding index and modifying the source variables so this will make it more clear. qemu: domain: Add vcpu private data structure Members will be added in follow-up patches. 2016-07-11 Peter Krempa conf: Add private data for virDomainVcpuDef Allow to store driver specific data on a per-vcpu basis. Move of the virDomainDef*Vcpus* functions was necessary as virDomainXMLOptionPtr was declared below this block and I didn't want to split the function headers. 2016-07-11 Peter Krempa conf: convert def->vcpus to a array of pointers 2016-07-11 Peter Krempa tests: qemuxml2xml: Format status XML header dynamically Status XML tests were done by prepending a constant string to an existing XML. With the planned changes the header will depend on data present in the definition rather than just on the data that was parsed. The first dynamic element in the header will be the vcpu thread list. Reuse and rename qemuXML2XMLPreFormatCallback for gathering the relevant data when checking the active XML parsing and formating and pass the bitmap to a newly crated header generator. 2016-07-11 Peter Krempa conf: Don't report errors from virDomainDefGetVcpu Most callers make sure that it's never called with an out of range vCPU. Every other caller reports a different error explicitly. Drop the error reporting and clean up some dead code paths. conf: Rename virDomainVcpuInfoPtr to virDomainVcpuDefPtr conf: Extract code formatting vCPU info conf: Annotate that private data for objects are not copied Our copy functions format and parse XML thus are not able to copy data. Annotate the private data pointers that this is happening. 2016-07-11 Nishith Shah virsh: Introduce vshReadlineParse for improved auto-completion The new function works as expected, and matches the current level of autocomplete offered, along with several other improvements like quotes handling, multiple command completion and space handling. Now, it is easy to introduce options completer here. virsh: Add option to suppress error in various functions A bool 'report' has been introduced in various functions, which when set to true will produce the error it is suppposed to produce, and when false, will suppress the error. These functions are used in the next patch for auto-completion. virsh: Fix variable types in readline generators Use unsigned int for array indexes and size_t for length variables. 2016-07-11 Nishith Shah virsh: Break vshCmddefOptParse into helper functions Decompose vshCmddefOptParse into two helper functions, vshCmddefOptFill and vshCmddefCheckInternals. vshCmddefCheckInternals checks if the internal command definitions are correct or not. vshCmddefOptFill keeps track of the required options and mandatory arguments through opts_required and opts_need_arg. 2016-07-11 Fabian Freyer bhyve: implement virConnectGetDomainCapabilities 2016-07-11 Roman Bogorodskiy bhyve: fix bhyveargv2xml custom loader test Before pushing this test, I changed the appropriate args file to pet test-wrap-argv.pl, but forgot to change the xml file, so update it accordingly. 2016-07-10 Fabian Freyer bhyve: add tests for bhyveParseCommandLineString 2016-07-10 Fabian Freyer bhyve: implement argument parser for loader A simple getopt-based argument parser is added for the /usr/sbin/bhyveload command, loosely based on its argument parser. The boot disk is guessed by iterating over all disks and matching their sources. If any non-default arguments are found, def->os.bootloaderArgs is set accordingly, and the bootloader is treated as a custom bootloader. Custom bootloader are supported by setting the def->os.bootloader and def->os.bootloaderArgs accordingly grub-bhyve is also treated as a custom bootloader. Since we don't get the device map in the native format anyways, we can't reconstruct the complete boot order. While it is possible to check what type the grub boot disk is by checking if the --root argument is "cd" or "hd0,msdos1", and then just use the first disk found, implementing the grub-bhyve argument parser as-is in the grub-bhyve source would mean adding a dependency to argp or duplicating lots of the code of argp. Therefore it's not really worth implementing that now. 2016-07-10 Fabian Freyer bhyve: implement bhyve argument parser A simpe getopt-based argument parser is added for the /usr/sbin/bhyve command, loosely based on its argument parser, which reads the following from the bhyve command line string: * vm name * number of vcpus * memory size * the time offset (UTC or localtime) * features: * acpi * ioapic: While this flag is deprecated in FreeBSD r257423, keep checking for it for backwards compatibiility. * the domain UUID; if not explicitely given, one will be generated. * lpc devices: for now only the com1 and com2 are supported. It is required for these to be /dev/nmdm[\d+][AB], and the slave devices are automatically inferred from these to be the corresponding end of the virtual null-modem cable: /dev/nmdmA <-> /dev/nmdmB * PCI devices: * Disks: these are numbered in the order they are found, for virtio and ahci disks separately. The destination is set to sdX or vdX with X='a'+index; therefore only 'z'-'a' disks are supported. Disks are considered to be block devices if the path starts with /dev, otherwise they are considered to be files. * Networks: only tap devices are supported. Since it isn't possible to tell the type of the network, VIR_DOMAIN_NET_TYPE_ETHERNET is assumed, since it is the most generic. If no mac is specified, one will be generated. 2016-07-10 Fabian Freyer bhyve: implement virConnectDomainXMLFromNative First, remove escaped newlines and split up the string into an argv-list for the bhyve and loader commands, respectively. This is done by iterating over the string splitting it by newlines, and then re-iterating over each line, splitting it by spaces. Since this code reuses part of the code of qemu_parse_command.c (in bhyveCommandLine2argv), add the appropriate copyright notices. 2016-07-10 Fabian Freyer gnulib: add getopt module Unconditionally use gnulib's getopt module. This is needed by the bhyve driver to provide a reentrant interface for getopt. Several gnulib headers rely on features.h being included by ctype.h to provide __GNUC_PREREQ, but on systems without glibc, this is not provided. In these cases __GNUC_PREREQ gets redefined to 0, which causes build errors from checks in src/internal.h. Therefore, define __GNUC_PREREQ as early as possible. config-post.h is probably the first header that is included, before any other headers. 2016-07-09 Marc Hartmayer tests: Add test cases for the empty bitmap As the empty bitmap exists, we should also test it. This patch adds test cases for the procedures 'virBitmapNextSetBit', 'virBitmapLastSetBit', 'virBitmapNextClearBit'. Tested-by: Sascha Silbe Reviewed-by: Sascha Silbe Reviewed-by: Boris Fiuczynski 2016-07-09 Marc Hartmayer util: bitmap: Mention the size == 0 handling As there is an explicit constructor for the special case of empty bitmaps, we should mention that the generic constructors rejects the creation of empty bitmaps. Reviewed-by: Boris Fiuczynski Reviewed-by: Sascha Silbe Reviewed-by: Bjoern Walk 2016-07-09 Marc Hartmayer util: bitmap: clarify virBitmapLastSetBit() behavior for empty bitmaps Before the variable 'bits' was initialized with 0 (commit 3470cd860d517760b13e26d97b6a842ff72687a1), the following bug was possible. A function call with an empty bitmap leads to undefined behavior. Because if 'bitmap->map_len == 0' 'unusedBits' will be <= 0 and 'sz == 1'. So the non global and non static variable 'bits' would have never been set. Consequently the check 'bits == 0' results in undefined behavior. This patch clarifies the current version of the function by handling the empty bitmap explicitly. Also, for an empty bitmap there is obviously no bit set so we can just return -1 (indicating no bit set) right away. The explicit check for 'bits == 0' after the loop is unnecessary because we only get to this point if no set bit was found. Reviewed-by: Boris Fiuczynski Reviewed-by: Sascha Silbe Reviewed-by: Bjoern Walk 2016-07-09 Fabian Freyer tests: env perl shebang for test-wrap-argv.pl On some systems perl is not necessarily in /usr/bin/perl. Use the perl version in the PATH instead. 2016-07-08 Jiri Denemark qemu: Drop useless SPICE migration code The spiceMigration flag will never be true if there is no SPICE graphics configured for the domain. https://bugzilla.redhat.com/show_bug.cgi?id=1151723 2016-07-08 Jiri Denemark qemu: Properly reset spiceMigration flag Otherwise migration during which we didn't send client_migrate_info QMP command will get stuck waiting for SPICE migration to finish if libvirtd sent the QMP command in a previous migration attempt. Broken by bd7c8a69. https://bugzilla.redhat.com/show_bug.cgi?id=1151723 2016-07-07 Daniel P. Berrange virtlogd: increase max file size to 2 MB People debugging guest OS boot processes and reported that the default 128 KB size is too small to capture an entire boot up sequence. Increase the default size to 2 MB which should allow capturing a full boot up even with verbose debugging. virtlogd: make max file size & number of backups configurable Currently virtlogd has a hardcoded max file size of 128kb and max of 3 backups. This adds two new config parameters to /etc/libvirt/virtlogd.conf to let these be customized. 2016-07-07 Peter Krempa qemu: caps: Always assume QEMU_CAPS_SMP_TOPOLOGY Support for SMP topology was added by qemu commit dc6b1c09849484fbbc50 prior to 0.12.0, our minimum supported qemu version. $ git describe --tags dc6b1c09849484fbbc50803307e4c7a3d81eab62 v0.11.0-rc0-449-gdc6b1c0 $ git describe --tags --contains dc6b1c09849484fbbc50803307e4c7a3d81eab v0.12.0-rc0~1477 2016-07-07 Michal Privoznik virDomainHostdevDefFree: Don't leak privateData After 27726d8c21 a privateData is allocated in virDomainHostdevDefAlloc(). However, the counter part - freeing them in Free() is missing which leads to the following memory leak: ==6489== 24 bytes in 1 blocks are definitely lost in loss record 684 of 1,003 ==6489== at 0x4C2C070: calloc (vg_replace_malloc.c:623) ==6489== by 0x54B7C94: virAllocVar (viralloc.c:560) ==6489== by 0x5517BE6: virObjectNew (virobject.c:193) ==6489== by 0x1B400121: qemuDomainHostdevPrivateNew (qemu_domain.c:798) ==6489== by 0x5557B24: virDomainHostdevDefAlloc (domain_conf.c:2152) ==6489== by 0x5575578: virDomainHostdevDefParseXML (domain_conf.c:12709) ==6489== by 0x5582292: virDomainDefParseXML (domain_conf.c:16995) ==6489== by 0x5583C98: virDomainDefParseNode (domain_conf.c:17470) ==6489== by 0x5583B07: virDomainDefParse (domain_conf.c:17417) ==6489== by 0x5583B95: virDomainDefParseFile (domain_conf.c:17441) ==6489== by 0x55A3F24: virDomainObjListLoadConfig (virdomainobjlist.c:465) ==6489== by 0x55A43E6: virDomainObjListLoadAllConfigs (virdomainobjlist.c:596) 2016-07-07 Paolo Bonzini qemu: generate -display none This is preferrable to -nographic which (in addition to disabling graphics output) redirects the serial port to stdio and on OpenBIOS enables the firmware's serial console. qemu: detect -display Add a new capability for the -display command line option, which has been present since QEMU 1.0. 2016-07-07 Peter Krempa conf: Isolate virDomainLiveConfigHelperMethod to libxl only Libxl is the last user and I don't have the toolchain prepared to compile the libxl driver. Move it to the libxl driver to avoid having to refactor the code. conf: Don't use virDomainLiveConfigHelperMethod in virDomainObjGetMetadata Few arguments of the function are not necessary any more which leads to some cleanups. The 'uri' argument had a stray ATTRIBUTE_UNUSED. conf: Don't use virDomainLiveConfigHelperMethod in virDomainObjSetMetadata openvz: Remove use of virDomainLiveConfigHelperMethod 2016-07-07 Peter Krempa lxc: Synchronize implementation of qemuDomainSetMemoryParameters The impls are identical and I don't have a reasonable idea where to extract it. This also kills yet another use of virDomainLiveConfigHelperMethod. 2016-07-07 Peter Krempa qemu: driver: Make name of QEMU_SET_MEM_PARAMETER more universal Use a VIR_ prefix even when it's a local helper macro. It will be later synced with the LXC implementation. 2016-07-07 Michal Privoznik spec: Move virt-admin into its own package The new package for the virt-admin binary is libvirt-admin. spec: Split libvirt-client Currently, we have libvirt-client library which serves as a collection point for all the libraries and client binaries we have. Therefore we have couple of silly dependencies, for instance libvirt-daemon depends on libvirt-client. Only because the shared library is in the client package. To solve this, new package libvirt-libs is introduced where all the libraries are going to live. The client package is then set to depend on this new package, just like the rest of packages that suffer the same problem. 2016-07-04 Erik Skultety virlog: Introduce virLogFilterListFree This is just a convenience method for discarding a list of filters instead of using a 'for' loop everywhere. It is safe to pass -1 as the number of elements in the list as well as passing NULL as list reference. virlog: Introduce virLogFilterFree Provide a separate method to free a logging filter object. This will come handy once a method to create an individual logging filter object is introduced. virlog: Introduce virLogOutputListFree This is just a convenience method for discarding a list of outputs instead of using a 'for' loop everywhere. It is safe to pass -1 as the number of elements in the list as well as passing NULL as list reference. virlog: Introduce virLogOutputFree Provide a separate method to free a logging output object. This will come handy once a method to create an individual logging output object is introduced. virlog: Convert virLogFilters to a list of pointers to filters Same as with outputs; since the operations will be further divided into smaller tasks, creating a filter will become a separate operation that will return a reference to a newly created filter. virlog: Convert virLogOutputs to a list of pointers to outputs Right now, we define outputs one after another. However, the correct flow should be to define a set of outputs as a whole unit. Therefore each output should be first created, placed into an array/list and the list will be defined. Output creation should be a separate operation, so an output will be returned by a reference. From that perspective, it makes perfect sense to only store pointers to actual outputs. virlog: Return void instead of int in virLogReset methods In this particular case, reset is meant as clearing the whole list of outputs/filters, not resetting it to a predefined default setting. Looking at it from that perspective, returning the number of records removed doesn't help the caller in any way (not that any of the callers would actually check for it). Well, callers could detect an error from the number of successfully removed records, but the only thing that can fail in virLogReset is force closing a file descriptor in which case the error isn't propagated back to virLogReset anyway. Conclusion: there is no practical use for having a return type of 'int' rather than 'void' in this case. 2016-07-04 Andrea Bolognani qemu: Memory locking is only required for KVM guests on ppc64 Due to the way the hardware works, KVM on ppc64 always requires memory locking; however, that is not the case for non-KVM ppc64 guests, eg. ppc64 guests that are running on x86_64 with TCG. Only require memory locking for ppc64 guests if they are using KVM or, as it's the case for all architectures, they have host devices assigned using VFIO. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1350772 2016-07-04 Martin Kletzander maint: update to latest gnulib Fix mingw build regarding rpl_{printf,scanf} symbols. 2016-07-02 Laine Stump qemu: support setting host-side IP addresses/routes For type='ethernet' interfaces only. (This patch had been pushed earlier in commit 0b4645a7e061abc8a4be71fe89865cf248ce6e56, but was reverted in commit 84d47a3cce71175bc1d8af596f835f66f38a190c because it had been accidentally pushed during the freeze for release 2.0.0) 2016-07-02 Laine Stump lxc: support setting host-side IP addresses/routes (This patch had been pushed earlier in commit cd5c9f21ded4f8e6216eba02b8795f70503ab404, but was reverted in commit 1549f16832ca23918b651fb4985a2e9f257cd429 because it had been accidentally pushed during the freeze for release 2.0.0) 2016-07-02 Laine Stump util: support setting peer for virNetDevIPInfo addresses This will apply to any IP address setting that uses virNetDevIPInfoAddToDev() (which so far is only the guest-side of LXC type='ethernet' interfaces). (This patch had been pushed earlier in commit cb20f989df393ec97ba65afb06089d0ab87af484, but was reverted in commit cba06aea8d500d4ea7e2f40272be484027dd3e4a because it had been accidentally pushed during the freeze for release 2.0.0) 2016-07-02 Laine Stump conf: support host-side IP/route information in This is place as a sub-element of , where other aspects of the host-side connection to the network device are located (network or bridge name, udp listen port, etc). It's a bit odd that the interface we're configuring with this info is itself named in , but that ship sailed long ago: In practice, this will likely only be useful for type='ethernet', so its presence in any other type of interface is currently forbidden in the generic device Validate function (but it's been put into the general population of virDomainNetDef rather than the ethernet-specific union member so that 1) we can more easily add the capability to other types if needed, and 2) we can retain the info when set to an invalid interface type all the way through to validation and report a proper error, rather than just ignoring it (which is currently what happens for many other type-specific settings). (NB: The already-existing configuration of IP info for the guest-side of interfaces is in subelements directly under , and the name of the guest-side interface (when configurable) is in ). (This patch had been pushed earlier in commit fe6a77898a38f491403a70cc49925a584101daee, but was reverted in commit d658456530c1010a49f45865613ed361a0fcc5b4 because it had been accidentally pushed during the freeze for release 2.0.0) 2016-07-02 Vasiliy Tolstov conf: allow setting peer address in element of The peer attribute is used to set the property of the same name in the interface IP info: ... ... Note that this element is used to set the IP information on the *guest* side interface, not the host side interface - that will be supported in an upcoming patch. (This patch now has quite a history: it was originally pushed in commit 690969af, which was subsequently reverted in commit 1d14b13f, then reworked and pushed (along with a lot of other related/supporting patches) in commit 93135abf1; however *that* commit had been accidentally pushed during dev. freeze for release 2.0.0, so it was again reverted in commit f6acf039f0). 2016-07-02 Laine Stump util: new function virNetDevIPInfoAddToDev This patch takes the code out of lxcContainerRenameAndEnableInterfaces() that adds all IP addresses and IP routes to the interface, and puts it into a utility function virNetDevIPInfoAddToDev() in virnetdevip.c so that it can be used by anyone. One small change in functionality - lxcContainerRenameAndEnableInterfaces() previously would add all IP addresses to the interface while it was still offline, then set the interface online, and then add the routes. Because I don't want the utility function to set the interface online, I've moved this up so the interface is first set online, then IP addresses and routes are added. This is the same order that the network service from initscripts (in ifup-ether) does it, so it shouldn't pose any problem (and hasn't, in the tests that I've run). (This patch had been pushed earlier in commit f1e0d0da11c473905470c28a6488bf57d9d0ae6e, but was reverted in commit 05eab47559950403aa67d18b098273269ae6916e because it had been accidentally pushed during the freeze for release 2.0.0) 2016-07-01 John Ferlan qemu: Introduce helper qemuDomainSecretDiskCapable Introduce a helper to help determine if a disk src could be possibly used for a disk secret... Going to need this for hot unplug. encryption: Add and to encryption For a luks device, allow the configuration of a specific cipher to be used for encrypting the volume. encryption: Add luks parsing for storageencryption Add parse and format of the luks/passphrase secret including tests for volume XML parsing. 2016-07-01 John Ferlan util: Add 'usage' for encryption In order to use more common code and set up for a future type, modify the encryption secret to allow the "usage" attribute or the "uuid" attribute to define the secret. The "usage" in the case of a volume secret would be the path to the volume as dictated by the backwards compatibility brought on by virStorageGenerateQcowEncryption where it set up the usage field as the vol->target.path and didn't allow someone to provide it. This carries into virSecretObjListFindByUsageLocked which takes the secret usage attribute value from from the domain disk definition and compares it against the usage type from the secret definition. Since none of the code dealing with qcow/qcow2 encryption secrets uses usage for lookup, it's a mostly cosmetic change. The real usage comes in a future path where the encryption is expanded to be a luks volume and the secret will allow definition of the usage field. This code will make use of the virSecretLookup{Parse|Format}Secret common code. 2016-07-01 John Ferlan conf: Add new secret type "passphrase" Add a new secret type known as "passphrase" - it will handle adding the secret objects that need a passphrase without a specific username. The format is: ... ... mumblyfratz 2016-07-01 John Ferlan conf: No need to check for usage fields during Format Since the virSecretDefParseUsage ensures each of the fields is present, no need to check during virSecretDefFormatUsage (also virBufferEscapeString is a no-op with a NULL argument). 2016-07-01 Brandon Bennett Allow custom metadata in network configuration XML This replicates the metadata field found in the domain configuration and adds it to the network configuration XML. 2016-07-01 Laine Stump util: new function virXMLNodeSanitizeNamespaces() This is a generic version of virDomainDefMetadataSanitize() - the same functionality is now needed for network metadata. docs: remove outdated suggestion to make patches with "diff -urp"/"git diff" I can't think of any good reason to do either of those, and having the examples there will just lead to unusable patch emails from people who can't be bothered to read the entire page. 2016-07-01 Andrea Bolognani util: hostcpu: Only define /dev/kvm path once Remove the local kvmpath variable from virHostCPUGetThreadsPerSubcore() and use the file-global KVM_DEVICE define instead. 2016-07-01 Ján Tomko examples: check asprintf return value in client_info.c On error, asprintf returns -1 and the contents of the string pointer is undefined. In the rest of the libvirt code, the virAsprintf wrapper takes care of that. Check the return value and report a generic error, since we purposefully avoid linking to virutil. 2016-07-01 Daniel P. Berrange mingw-libvirt.spec.in: add perl + perl(Getopt::Long) BRs The default Fedora build roots for f25 and newer no longer include perl. We must thus explicitly ask for it as the RPC gendispatch.pl program needs it, and the Getopt::Long module. Do this unconditionally since it isn't harmful for older Fedora 2016-07-01 Daniel P. Berrange mingw-libvirt.spec.in: fix packaging of admin API and other bugs When the admin API was enabled no entries were added to the file list. The virt-host-validate binary is also no longer built on win32 2016-07-01 Cole Robinson events: Rename argument uuid->key Since it's not strictly a uuid anymore events: Add explicit lookup 'key' value This allows event implementations to match on something other than an object's uuid, like nodedev or interface objects which don't have a uuid. events: Pass in UUID as a string This should not have any functional difference, it's just a step towards matching on non-uuid string keys events: Cleanup callback variable name In every other instance virObjectEventCallbackPtr is named 'cb', and in other code 'event' usually means a virObjectEventPtr events: Add virObjectEventCallbackFree events: Privatize virObjectEventCallback It's only used in object_event.c, so remove it from the header. We need to move the _virObjectEventCallback definition earlier as a result. 2016-07-01 Ján Tomko Introduce virDomainUSBDeviceDefForeach A helper that will execute a callback on every USB device in the domain definition. With an ability to skip USB hubs, since we will want to treat them differently in some cases. 2016-07-01 Ján Tomko Add USB addresses to qemuhotplug test cases This test assumes the XML will be the same after formatting. Add USB addresses to it to keep it working when we autoassign them. 2016-07-01 Jiri Denemark qemu: Avoid needless copies of static strings qemu: Drop emitBootindex parameter 2016-07-01 Jiri Denemark qemu: Use bootindex whenever possible I'm not sure why our code claimed "-boot menu=on" cannot be used in combination with per-device bootindex, but it was proved wrong about four years ago by commit 8c952908. Let's always use bootindex when QEMU supports it. https://bugzilla.redhat.com/show_bug.cgi?id=1323085 2016-07-01 Jiri Denemark qemu: Remove redundant parameter in virQEMUCapsFillDomainCaps virttype is already included in domCaps, no need to pass it separately. domaincapstest: Don't read data from host virQEMUCapsFillDomainCaps would use virHostCPUGetKVMMaxVCPUs for KVM domains. cpu: Drop NR_DRIVERS macro cpu: Drop generic driver Pretending (partial) support for something we don't understand is risky. Reporting a failure is much better. Post-release version bump to 2.1.0 2016-07-01 Daniel Veillard Libvirt 2.0.0 release * docs/news.html.in: update documentation * po/*.po*: regenerate 2016-06-30 Martin Kletzander dist: Speed up distribution compression This almost reverts b7200d723648. The size is increased from 11M to 13M and the compression is sped up from 2 minutes to 17 seconds. The compression level is removed because -9 doesn't allow multiple threads to be spawned. Effectively speeds up distcheck as well. 2016-06-30 Peter Krempa conf: Don't free the constructed string in virDomainGetBlkioParametersAssignFromDef virTypedParameterAssign steals the string rather than copying it into the typed parameter and thus freeing it leads to a crash when attempting to serialize the results. This was introduced in commit 9f50f6e2 and later made an universal helper in 32e6339c. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1351473 2016-06-30 Peter Krempa conf: def: Avoid unnecessary allocation of 'perf' events definition Some code paths already assume that it is allocated since it was always allocated by virDomainPerfDefParseXML. Make it member of virDomainDef directly so that we don't have to allocate it all the time. This fixes crash when attempting to connect to an existing process via virDomainQemuAttach since we would not allocate it in that code path. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1350688 2016-06-30 Jiri Denemark docs: Warn against locked memory limit too high https://bugzilla.redhat.com/show_bug.cgi?id=1046833 2016-06-30 Marc Hartmayer tests: Add test cases for SCSI disk hot-plug with QEMU Verify that SCSI controllers get created automatically when a SCSI disk is hot-plugged to a domain that doesn't have a matching SCSI controller defined already. Reviewed-by: Boris Fiuczynski 2016-06-30 Marc Hartmayer qemu: SCSI hostdev hot-plug: Fix automatic creation of SCSI controllers Ensure that the given controller and all controllers with a smaller index exist; there must not be any missing index in between. Reviewed-by: Boris Fiuczynski Reviewed-by: Bjoern Walk 2016-06-30 Marc Hartmayer qemu: hot-plug: Fix broken SCSI disk hot-plug The commit "qemu: hot-plug: Assume support for -device in qemuDomainAttachSCSIDisk" dropped the code for the automatic SCSI controller creation used in SCSI disk hot-plugging. If we are hot-plugging a SCSI disk to a domain and there is no proper SCSI controller defined, it results in an "error: internal error: Could not find scsi controller with index X required for device" error. For that reason reverting a hunk of the commit d4d32005d6e8b2cc0a2f26b483ca1de10171db6d. This patch also adds an extra comment to the code to clarify the loop. Reviewed-by: Boris Fiuczynski Reviewed-by: Bjoern Walk 2016-06-30 Jiri Denemark qemu: Let empty default VNC password work as documented CVE-2016-5008 Setting an empty graphics password is documented as a way to disable VNC/SPICE access, but QEMU does not always behaves like that. VNC would happily accept the empty password. Let's enforce the behavior by setting password expiration to "now". https://bugzilla.redhat.com/show_bug.cgi?id=1180092 2016-06-30 Jiri Denemark qemu: Use proper async job to refresh virtio channels 2016-06-29 Jean-Marc Liger libvirt.spec.in: Use libnl-devel for RHEL-6 RHEL-6 still needs to use libnl instead of libnl3, so re-add the spec conditional mistakenly removed in commit 3694e038fd12825042cca6204f7aa7322e2ed9c9 Author: Daniel P. Berrange Date: Wed May 4 15:43:08 2016 +0100 libvirt.spec.in: drop Fedora < 20 and RHEL < 6 2016-06-29 Erik Skultety admin: fix virt-admin startup crash by calling virAdmInitialize Similarly to what virsh virt-login-shell do, call virAdmInitialize prior to initializing an event loop and initializing the error handler. Commit 97973ebb7 described and fixed an identical issue for libvirt_lxc. Since virAdmInitialize becomes a public API after applying this patch, the symbol is also added to public syms and the doc string of the method is slightly enhanced analogically to virInitialize. 2016-06-29 Ján Tomko Fix possible invalid read in adminClientGetInfo virNetServerClientGetInfo returns the client's remote address as a string, which is a part of the client object. Use VIR_STRDUP to make a copy which can be freely accessed even after the virNetServerClient object is unlocked. To reproduce, put a sleep between virObjectUnlock in virNetServerClientGetInfo and virTypedParamsAddString in adminClientGetInfo, then close the queried connection during that sleep. 2016-06-28 Michal Privoznik virStorageTranslateDiskSourcePool: Avoid double free https://bugzilla.redhat.com/show_bug.cgi?id=1316370 Consider the following disk for a domain:
Now, startupPolicy is currently not allowed for iscsi disks, so one would expect an error message to be thrown. But what a surprise is waiting for users if they try to start up such domain: ==15724== Invalid free() / delete / delete[] / realloc() ==15724== at 0x4C2B1F0: free (vg_replace_malloc.c:473) ==15724== by 0x54B7A69: virFree (viralloc.c:582) ==15724== by 0x552DC90: virStorageAuthDefFree (virstoragefile.c:1549) ==15724== by 0x552F023: virStorageSourceClear (virstoragefile.c:2055) ==15724== by 0x552F054: virStorageSourceFree (virstoragefile.c:2067) ==15724== by 0x55556AA: virDomainDiskDefFree (domain_conf.c:1562) ==15724== by 0x5557ABE: virDomainDefFree (domain_conf.c:2547) ==15724== by 0x1B43CC42: qemuProcessStop (qemu_process.c:5918) ==15724== by 0x1B43BA2E: qemuProcessStart (qemu_process.c:5511) ==15724== by 0x1B48993E: qemuDomainObjStart (qemu_driver.c:7050) ==15724== by 0x1B489B9A: qemuDomainCreateWithFlags (qemu_driver.c:7104) ==15724== by 0x1B489C01: qemuDomainCreate (qemu_driver.c:7122) ==15724== Address 0x21cfbb90 is 0 bytes inside a block of size 48 free'd ==15724== at 0x4C2B1F0: free (vg_replace_malloc.c:473) ==15724== by 0x54B7A69: virFree (viralloc.c:582) ==15724== by 0x552DC90: virStorageAuthDefFree (virstoragefile.c:1549) ==15724== by 0x12D1C8D4: virStorageTranslateDiskSourcePool (storage_driver.c:3475) ==15724== by 0x1B4396E4: qemuProcessPrepareDomain (qemu_process.c:4896) ==15724== by 0x1B43B880: qemuProcessStart (qemu_process.c:5466) ==15724== by 0x1B48993E: qemuDomainObjStart (qemu_driver.c:7050) ==15724== by 0x1B489B9A: qemuDomainCreateWithFlags (qemu_driver.c:7104) ==15724== by 0x1B489C01: qemuDomainCreate (qemu_driver.c:7122) ==15724== by 0x561CA97: virDomainCreate (libvirt-domain.c:6787) ==15724== by 0x12B6FD: remoteDispatchDomainCreate (remote_dispatch.h:4116) ==15724== by 0x12B61A: remoteDispatchDomainCreateHelper (remote_dispatch.h:4092) The problem is, in virStorageTranslateDiskSourcePool disk def->src->auth is freed, but the pointer is not set to NULL. So later, when qemuProcessStop starts to free the domain definition, virStorageAuthDefFree() tries to free the memory again, instead of jumping out immediately. 2016-06-28 Jaroslav Suchanek logging: fixing log level initialization from cmdline Reorder code for setting default log level from cmdline prior initialization of log outputs. Thus the --verbose option is reflected. This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1325072 2016-06-28 Jiri Denemark qemuDomainDeviceDefValidate: Drop unused qemuCaps 2016-06-28 Andrea Bolognani vz: Fix indentation in prlsdkGetNetAddresses() 2016-06-27 Olga Krishtal vz: fix build for virNetDev* changes Patch fixes vz build after changes in IP-related netdev functions(cf0568b0, fbc1843d). 2016-06-27 Ján Tomko Revert "util: new function virNetDevIPInfoAddToDev" This reverts commit f1e0d0da11c473905470c28a6488bf57d9d0ae6e. This feature was accidentally pushed in the feature freeze. 2016-06-27 Ján Tomko Revert "conf: allow setting peer address in element of " This reverts commit 93135abf1454d8a1c8542e8c951ed615305ffa24. This feature was accidentally pushed in the feature freeze. 2016-06-27 Ján Tomko Revert "conf: support host-side IP/route information in " This reverts commit fe6a77898a38f491403a70cc49925a584101daee. This feature was accidentally pushed in the feature freeze. 2016-06-27 Ján Tomko Revert "util: support setting peer for virNetDevIPInfo addresses" This reverts commit cb20f989df393ec97ba65afb06089d0ab87af484. This feature was accidentally pushed in the feature freeze. 2016-06-27 Ján Tomko Revert "lxc: support setting host-side IP addresses/routes" This reverts commit cd5c9f21ded4f8e6216eba02b8795f70503ab404. This feature was accidentally pushed in the feature freeze. 2016-06-27 Ján Tomko Revert "qemu: support setting host-side IP addresses/routes" This reverts commit 0b4645a7e061abc8a4be71fe89865cf248ce6e56. This feature was accidentally pushed in the feature freeze. 2016-06-27 Andrea Bolognani Clean up after virNetDevIP creation Commit cf0568b0af4e moved a bunch of functions from virNetDev to the more specific virNetDevIP; however, not all of the existing uses were moved properly, causing build failures on FreeBSD. Complete the transition to the new names and drop the obsolete declarations from the header file while at it. 2016-06-27 Andrea Bolognani util: netdevip: Include vircommand.h Not including the header causes util/virnetdevip.c:520:5: error: unknown type name 'virCommandPtr'; did you mean 'virCondPtr'? virCommandPtr cmd = NULL; ^~~~~~~~~~~~~ and plenty more similar failures when compiling on FreeBSD. 2016-06-27 Michal Privoznik libvirtd.conf: Fix invalid default of max_anonymous_clients https://bugzilla.redhat.com/show_bug.cgi?id=1343442 When a client connects, it is placed into a queue. As soon as it authenticate, it is taken out of that queue and placed into a different one. Now, we have a setting in the daemon config file that allows users to control the length of the queue of yet not authenticated clients. By default, it has a value 20 but in the description to the config knob we clam it's zero. 2016-06-27 Laine Stump qemu: support setting host-side IP addresses/routes For type='ethernet' interfaces only. lxc: support setting host-side IP addresses/routes util: support setting peer for virNetDevIPInfo addresses This will apply to any IP address setting that uses virNetDevIPInfoAddToDev() (which so far is only the guest-side of LXC type='ethernet' interfaces). 2016-06-27 Laine Stump conf: support host-side IP/route information in This is place as a sub-element of , where other aspects of the host-side connection to the network device are located (network or bridge name, udp listen port, etc). It's a bit odd that the interface we're configuring with this info is itself named in , but that ship sailed long ago: In practice, this will likely only be useful for type='ethernet', so its presence in any other type of interface is currently forbidden in the generic device Validate function (but it's been put into the general population of virDomainNetDef rather than the ethernet-specific union member so that 1) we can more easily add the capability to other types, and 2) we can retain the info when set to an invalid interface type all the way through to validation and report a proper error, rather than just ignoring it (which is currently what happens for many other type-specific settings). (NB: The already-existing configuration of IP info for the guest-side of interfaces is in subelements directly under , and the name of the guest-side interface (when configurable) is in ). 2016-06-27 Vasiliy Tolstov conf: allow setting peer address in element of The peer attribute is used to set the property of the same name in the interface IP info: ... ... Note that this element is used to set the IP information on the *guest* side interface, not the host side interface - that will be supported in an upcoming patch. (This is an updated *re*-commit of commit 690969af, which was subsequently reverted in commit 1d14b13f). 2016-06-27 Laine Stump util: new function virNetDevIPInfoAddToDev This patch takes the code out of lxcContainerRenameAndEnableInterfaces() that adds all IP addresses and IP routes to the interface, and puts it into a utility function virNetDevIPInfoAddToDev() in virnetdevip.c so that it can be used by anyone. One small change in functionality - lxcContainerRenameAndEnableInterfaces() previously would add all IP addresses to the interface while it was still offline, then set the interface online, and then add the routes. Because I don't want the utility function to set the interface online, I've moved this up so the interface is first set online, then IP addresses and routes are added. This is the same order that the network service from initscripts (in ifup-ether) does it, so it shouldn't pose any problem (and hasn't, in the tests that I've run). 2016-06-27 Laine Stump lxc: move debug/error log when adding IP addresses to virNetDevIPAddrAdd It makes more sense to have the logging at the lower level so other callers can share the goodness. While removing so much stuff from / touching so many lines in lxcContainerRenameAndEnableInterfaces() (which used to have this debug/error logging), label names were changed and it was updated to use the now-more-common method of initializing ret to -1 (failure), then setting to 0 right before the cleanup label. 2016-06-27 Laine Stump conf: clean up after adding calls to virNetDevIPInfo helpers virDomainNetIPInfoParseXML() and virDomainNetIPInfoFormat() are no longer "unused", so we can now remove the "ATTRIBUTE_UNUSED" from their definitions, since virDomainNetIPInfoFormat() is now the only caller of virDomainNetIPsFormat() and virDomainNetRoutesFormat(), those two functions can simply be subsumed into virDomainNetIPInfoFormat(). 2016-06-27 Laine Stump qemu: forbid setting guest-side IP address/route info of libvirt's qemu driver doesn't have direct access to the config on the guest side of a network interface, and currently doesn't have any method in place to even inform the guest of the desired config. In the future, an unenforceable attempt to set the guest-side IP info could be made by adding a static host entry to the appropriate dnsmasq configuration (or changing the default dhcp client address on the qemu commandline for type='user' interfaces), or enhancing the guest agent to allow setting an IP address, but for now it can't have any effect, and we don't want to give the illusion that it does. To prevent the "disappearance" of any existing configs with ip address/route info (due to parser failure), this check is added in the newly implemented qemuDomainDeviceDefValidate(), which is only called when a domain is defined or started, *not* when it is reread from disk at libvirtd startup. 2016-06-27 Laine Stump conf: use virNetDevIPInfo for guest-side config All the same information was already there, just in slightly different places in the virDomainNetDef. 2016-06-27 Laine Stump conf: use virNetDevIPInfo in virDomainHostdevCaps a.k.a. . This replaces the existing nips, ips, nroutes, and routes with a single virNetDevIPInfo, and simplifies the code by calling that object's parse/format/clear functions instead of open coding. 2016-06-27 Laine Stump conf: single object containing list of IP addresses, list of routes There are currently two places in the domain where this combination is used, and there is about to be another. This patch puts them together for brevity and uniformity. As with the newly-renamed virNetDevIPAddr and virNetDevIPRoute objects, the new virNetDevIPInfo object will need to be accessed by a utility function that calls low level Netlink functions (so we don't want it to be in the conf directory) and will be called from multiple hypervisor drivers (so it can't be in any hypervisor directory); the most appropriate place is thus once again the util directory. The parse and format functions are in conf/domain_conf.c because only the domain XML (i.e. *not* the network XML) has this exact combination of IP addresses plus routes. Note that virDomainNetIPInfoFormat() will end up being the only caller to virDomainNetRoutesFormat() and virDomainNetIPsFormat(), so it will just subsume those functions in a later patch, but we can't do that until they are no longer called. (It would have been nice to include the interface name within the virNetDevIPInfo object (with a slight name change), but that can't be done cleanly, because in each case the interface name is provided in a different place in the XML relative to the routes and IP addresses, so putting it in this object would actually make the code more confused rather than simpler). 2016-06-27 Laine Stump util: move IP route & address object-related functions to virnetdevip.c These functions all need to be called from a utility function that must be located in the util directory, so we move them all into util/virnetdevip.[ch] now that it exists. Function and struct names were appropriately changed for the new location, but all code is unchanged aside from motion and renaming. 2016-06-27 Laine Stump util: new files virnetdevip.[ch] for IP-related netdev functions This patch splits virnetdev.[ch] into multiple files, with the new virnetdevip.[ch] containing all the functions related to setting and retrieving IP-related info for a device (both addresses and routes). 2016-06-27 Laine Stump conf/openvz: eliminate incorrect/undocumented use of When support for was added in commit 9a4b705f back in 2010, it erroneously looked at for a user-specified guest-side interface name. This was never documented though. (that attribute already existed at the time in the data.ethernet union member of virDomainNetDef, but apparently had no practical use - it was only used as a storage place for a NetDef's bridge name during qemuDomainXMLToNative(), but even then that was never used for anything). When support for similar guest-side device naming was added to the lxc driver several years later, it was put in a new subelement . In the intervening years, since there was no validation that ethernet.dev was NULL in the other drivers that didn't actually use it, innocent souls who were adding other features assuming they needed to account for non-NULL ethernet.dev when really they didn't, so little bits of the usual pointless cargo-cult code showed up. This patch not only switches the openvz driver to use the documented notation for naming the guest-side device (just in case anyone is still using the openvz driver), and logs an error if anyone tries to set for a type='ethernet' interface, it also removes the cargo-cult uses of ethernet.dev and , and eliminates if from the RNG and from virDomainNetDef. NB: I decided on this course of action after mentioning the inconsistency here: https://www.redhat.com/archives/libvir-list/2016-May/msg02038.html and getting encouragement do eliminate it in a later IRC discussion with danpb. 2016-06-27 Laine Stump qemu: eliminate memory leaks when converting NetDefs to type='ethernet' in qemuConnectDomainXMLToNative. This function was only accounting for about 1/10 of all the allocated items in the NetDef prior to memseting it to all 0's. On top of that, it was going to great pains to learn the name of the bridge device, but then never doing anything useful with it (just putting it into data.ethernet.dev, which is *never* used when building a qemu commandline). (I think this again all started off as code with good intentions, but it was never completed, and instead was just Frankensteinically cargo-culted into the odd mish mash we have today). The resulting code is much simpler, produces exactly the same output, and doesn't leak memory. 2016-06-27 Laine Stump qemu: don't set/clear NetDef IP addresses in qemuConnectDomainXMLToNative() This patch removes the expanded and duplicated code that all sprung out of two well-intentioned-but-useless settings of net->data.(bridge|ethernet).ipaddr. qemu has never supported even a single IP address in the interface config, much less a list of them. All of the instances of "clearing out the IP addresses" that are now in this function originated with commit d8dbd6 "Basic domain XML conversions for Xen/QEMU drivers" in May 2009, but even then the single "ipaddr" in the struct for type='ethernet' and type='bridge' wasn't used in the qemu driver (only in xen and openvz). Since then anyone who added a new interface type also tacked on another unnecessary clearing of ipaddr, and when it was made into a list of IPs (so far supported only by the LXC driver) this simple setting was turned into a loop (well, multiple loops) to clear them all. 2016-06-27 Laine Stump conf: new function virDomainNetDefClear We need to clear these out without freeing the object completely. 2016-06-27 Laine Stump lxc: use correct prefix when setting veth IP address Commit c9a641 (first appearred in 1.2.12) added support for setting the guest-side IP address of veth devices in lxc domains. Unfortunately, it hardcoded the assumption that the proper prefix for any IP address with no explicit prefix in the config should be "24"; that is only correct for class C IPv4 addresses, but not for any other IPv4 address, nor for any IPv6 address. The good news is that there is already a function in libvirt that will determine the proper default prefix for any IP address. This patch replaces the use of the ill-fated VIR_SOCKET_ADDR_DEFAULT_PREFIX with calls to virSocketAddrGetIPPrefix(). 2016-06-27 Laine Stump lxc: eliminate extraneous free of netDef->ifname_guest lxcContainerRenameAndEnableInterfaces() isn't making a copy of the interface's ifname_guest (into newname), it's just copying the pointer to it. This means that when it later calls VIR_FREE(newname), it's actually freeing up (and fortunately NULLing out, so at least we don't try to access free'd memory) netDef->ifname_guest. util: allow calling virSocketAddrGetIPPrefix with NULL netmask or address There are times when we don't have a netmask pointer to give to virSocketAddrGetIPPrefix() (e.g. the IP addresses in domain interfaces only have a prefix, no netmask), but it would have caused a segv if we called it with NULL instead of a pointer to a netmask. This patch qualifies the code that would use the netmask or address pointers to check for NULL first. tests: mock virNetDevSetIPAddress Now that we can include in tests, we could almost test XML that has an element in an interface. Except that the test fails when it tries to actually set the IP address for the interface's tap device. This patch mocks virNetDevSetIPAddress() to just return success. 2016-06-27 Laine Stump conf: clean up virDomainNetIPParseXML() Rearrange this function to be better organized and more correct: * the error codes were changed from the incorrect INVALID_ARG to XML_ERROR * prefix still isn't required, but if present it must be valid or an error will be logged. * don't emit a debug log just because prefix is missing - this is valid. * group everything related to setting prefix in one place rather than scattered through the function. 2016-06-27 Laine Stump global: consistently use IP rather than Ip in identifiers I'm tired of mistyping this all the time, so let's do it the same all the time (similar to how we changed all "Pci" to "PCI" awhile back). (NB: I've left alone some things in the esx and vbox drivers because I'm unable to compile them and they weren't obviously *not* a part of some API. I also didn't change a couple of variables named, e.g. "somethingIptables", because they were derived from the name of the "iptables" command) 2016-06-27 Laine Stump util: move virInterface(State|Link)/virNetDevFeature from conf to util These had been declared in conf/device_conf.h, but then used in util/virnetdev.c, meaning that we had to #include conf/device_conf.h in virnetdev.c (which we have for a long time said shouldn't be done. This caused a bigger problem when I tried to #include util/virnetdev.h in a file in src/conf (which is allowed) - for some reason the "device_conf.h: File not found" error. The solution is to move the data types and functions used in util sources from conf to util. Some names were adjusted during the move ("virInterface" --> "virNetDevIf", and "VIR_INTERFACE" --> "VIR_NETDEV_IF") 2016-06-27 Laine Stump util: move virNetDevLinkDump to virnetlink.c virNetDevLinkDump should have been in virnetlink.c, but that file didn't exist yet when the function was created. It didn't really matter until now - I found that having virnetlink.h included by virnetdev.h caused build problems when trying to #include virnetdev.h in a .c file in src/conf (due to missing directory in -I). Rather than fix that to further institutionalize the incorrect placement of this one function, this patch moves the function. 2016-06-26 Erik Skultety spec: distribute admin API within libvirt-client package With respect to to the following thread https://www.redhat.com/archives/libvir-list/2016-June/msg01822.html, until we introduce a new rpm package '-libs' that would allow us to drop daemon's dependency on the client package, distribute admin API related stuff within the client package (since it's the best analogy to the virsh client). examples: admin: Add some examples for the new admin APIs Some of the examples make use of asprintf and strtol functions (to keep things simple) which are prohibited to use within our code (enforced by syntax-check). Therefore besides adding some examples, this patch also updates cfg.mk to exclude examples directory from asprintf and strtol rules, as well as updates .gitignore to exclude all the new admin binaries created in the 'examples' dir. admin: enable both admin API functionality and tarball distribution This patch enables admin socket creation in daemon's code, bumps the library version in libvirt_admin_public.syms, and performs all necessary modifications to our makefiles so that admin API can finally be included in the tarball, and eventually become part of an rpm package (a patch later in this series). 2016-06-25 Nikolay Shirokovskiy vz: always pass graphics address to sdk We need this because apply graphics functions is used on update too. Also in case of NULL address resolve it to default instead of error. vz: support vnc password vz: remove exlicitly setting zeros in dumping graphics Allocation will do this job. Also we don't use the explicit setting in other places. vz: support attach/detach/update/ of graphics device Move graphic device config to post parse. This way we detect error on early stage and leverage checking on detach too. vz: move getting container video devices out from vnc code 2016-06-25 Nikolay Shirokovskiy vz: trustGuestRxFilters fixes First we need to always set value to vz sdk parameter so we can leverage setting code for device updates. This patch resolves tristate default to off implicitly. This is easier then extract default value from vz sdk itself. First current default is off too, second this approach is already taken for 'net->linkstate'. Second dump this option in domain xml. 2016-06-25 Nikolay Shirokovskiy vz: fix minor type safey issues with net union usage Fix net->data usage accordingly to type field. 2016-06-25 Nikolay Shirokovskiy vz: fix updating to no gateways Current code that pass gateways to vz sdk is not suitable for updates. If update has no gateways while we had them before we need to pass "" for vz sdk gateways to reset old value. The code definitely deserves its own function. Drop checks that skip setting gateways if network address is not set. Such a configuration is possible in vz sdk. 2016-06-25 Nikolay Shirokovskiy vz: dump route info in domain xml vz: dump ip addresses to domain xml vz: give nice report if network device not found vz: fix memory leaks in attach/detach functions vz: move disks checks to device post parse And reformat so that we don't have lengthy lines. Also simplify some checks. vz: leverage disks parameters check on disks updates too This is as easy as moving disks checks from domain post parse callback to device post parse callback. vz: add device updates vz: reuse edit config frame in for attach/detach functions Attach/detach functions for disk/net are quite trivial and typically call a few functions in begin/end edit frame. Having in mind update function too adding configuring for another device (like graphics) will introduce 3 trivial functions more. Let's replace current approach by attach/detach functions for device. vz: make prlsdkGetDisk more generic Current implementation works with hard disks only. This patch adds support for any disk device (cdroms and hdds right now). vz: remove disk cache mode hunk This code was added as a part of huge patch that moves driver from working with prlctl to vz sdk so there is no good explanation why this is done this way. The problem that it is not correct. vz sdk cache mode parameter affects all domain disks while this hunk resets its on every disk to a new value. 2016-06-25 Qiaowei Ren cpu_map.xml: add cmt/mbm feature to x86 Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family) introduced some PQos (Platform Qos) features, including CMT (Cache Monitoring technology) and MBM (Memory Bandwidth Monitoring), to monitor or control shared resource. This patch add them into x86 part of cpu_map.xml to be used for applications based on libvirt to get cpu capabilities. For example, Nova in OpenStack schedules guests based on the CPU features that the host has. 2016-06-25 Jiri Denemark cpu: Consolidate ARM drivers Both ARM and AArch64 drivers are exactly the same (modulo function names). Let's use just one driver for all ARM architectures. 2016-06-24 Roman Bogorodskiy util: fix build in virNetDevTapGetRealDeviceName Commit e81de04c switched virNetDevTapGetRealDeviceName() to use virDirOpen() instead of opendir(), however it mistakenly dropped DIR *dirp declaration, so restore that to fix build. 2016-06-24 John Ferlan storage: Introduce virStoragePoolObjBuildTempFilePath Create a function to return a temporary file path to be used in a mkostemp type call using the path to the stateDir + pool->def->name + vol->name 2016-06-24 Daniel P. Berrange Promote storage pool refresh lifecycle event to top level event The VIR_STORAGE_POOL_EVENT_REFRESHED constant does not reflect any change in the lifecycle of the storage pool. It should thus not be part of the storage pool lifecycle event set, but rather be a top level event in its own right. Thus we introduce VIR_STORAGE_POOL_EVENT_ID_REFRESH to replace it. 2016-06-24 John Ferlan util: Add 'luks' to the FileTypeInfo Add the ability to detect a luks encrypted device. util: Modify the FileTypeInfo to add a version size The version field historically has been a 4 byte data; however, an upcoming new type will use a 2 byte version. So let's adjust for that now. util: Introduce virReadBufInt16LE and virReadBufInt16BE In order to read 16 bits of data in the native format and convert add the 16 bit macros to match existing 32 and 64 bit code. 2016-06-24 John Ferlan qemu: Remove authdef from secret setup Rather than pass authdef, pass the 'authdef->username' and the '&authdef->secdef' Note that a username may be NULL. 2016-06-24 John Ferlan qemu: Change protocol parameter for secret setup Rather than assume/pass the protocol to the qemuDomainSecretPlainSetup and qemuDomainSecretAESSetup, set and pass the secretUsageType based on the src->protocol type. This will eventually be used by the virSecretGetSecretString call 2016-06-24 Shivaprasad G Bhat Document to not rely on virConnectGetMaxVcpus API The API virConnectGetMaxVcpus doesn't really reflect the actual usable number of cpus as the maximum limits can be different for kvm and/or qemu. So update the documentation to use virConnectGetDomainCapabilities() instead. qemu: check the kvm host cpu max limits in virConnectGetDomainCapabilities The qemu limit and host limit both should be considered for the domain vcpu max limits. 2016-06-24 John Ferlan qemu: Make qemuBuildSecretInfoProps global Need to create the object for a hotplug disk qemu: Remove type from qemuBuildSecretInfoProps It's just a constant "secret" string anyway 2016-06-24 Shivaprasad G Bhat Rename kvmGetMaxVCPUs() to virHostCPUGetKVMMaxVCPUs() This kvmGetMaxVCPUs() needs to be used at two different places so move it to utils with appropriate name and mark it as private global now. 2016-06-24 yuelongguang add help document relevant to default mapping of credentials to machines * src/util/virauthconfig.c 2016-06-24 Peter Krempa conf: Allow disks with identical WWN or serial Disallowing them broke a use case of testing multipath configurations for storage. Originally this was added as it was impossible to use certain /dev/disk-by... links but the disks worked properly. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1349895 2016-06-24 Ján Tomko build: increase xz compression level Increase the default compression level to 9 from 6. This also increases decompression memory requirements from 9 MB to 65 MB. Also turn on verbosity. 2016-06-24 Ján Tomko openvz: do not open-code STRSKIP Remove one more use of STREQLEN with strlen as its argument. 2016-06-24 Ján Tomko Fix error detection in virStorageBackendISCSIGetHostNumber In the unlikely case the iSCSI session path exists, but does not contain an entry starting with "target", we would silently use an initialized value. Rewrite the function to correctly report errors. 2016-06-24 Ján Tomko Replace some uses STREQLEN with STRPREFIX Do not call it with a magic constant matching the length of the pattern. 2016-06-24 John Ferlan docs: Fix whitespace in output Many moons ago, commit id '8d7800a55' adjusted the format of the output to add a space on the HEADER and the DATA... the docs weren't updated to reflect that... This makes that adjustment. 2016-06-24 Shivaprasad G Bhat qemu: Check for VFIO too where legacy passthrough is checked On PPC the legacy passthrough is not supported and only VFIO is supported. So, the checks at places to confirm if the host is passthrough capable checks only legacy, fix it. This is seen at only one place now. 2016-06-24 Ján Tomko virStorageBackendISCSIGetHostNumber: correctly use virDirOpen Incorrect conflict resolution in my commit e81de04c1 broke this. Prohibit opendir in syntax-check Prefer virDirOpen. Use virDirOpenQuiet Remove all the remaining usage of opendir. Introduce virDirOpenQuiet A helper function that does not report any errors. Use virDirOpenIfExists Use it instead of opendir everywhere we need to check for ENOENT. Add virDirOpenIfExists Just like virDirOpen, but it returns 0 without reporting an error on ENOENT. Use virDirOpen Switch from opendir to virDirOpen everywhere we need to report an error. Introduce virDirOpen A helper that calls opendir and reports an error if it fails. 2016-06-24 Cole Robinson storage: Fix coverity warning After commit e808d3f227 cbdata is always available here, so the check is pointless 2016-06-24 Martin Kletzander Don't allow raneming domains to empty strings It may cause unwanted behaviour (of course, is there any wanted one for that case?) so we should rather disable the possibility of doing so. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1320893 2016-06-24 Ján Tomko openvz: split single-line if Put the 'continue' statement on a separate line. 2016-06-24 Andrea Bolognani qemu: Use stricter checks in virQEMUCapsFillDomainDeviceDiskCaps() Unfortunately, we can't just call qemuDomainMachineIsPSeries() here, because we don't have a virDomainDef instance; that said, the open-coded check should match said function as closely as possible. qemu: Introduce qemuDomainMachineIsPSeries() This new function checks for both the architecture and the machine type, so we can use it instead of writing the same checks over and over again. qemu: Add architecture checks to qemuDomainMachineIsVirt() Remove all external architecture checks that have been made redundant by this change. qemu: Remove redundant arguments to qemuBuildSerialChrDeviceStr() Since we're already passing the full virDomainDef, it doesn't make sense to also pass def->os.arch and def->os.machine as separate arguments. 2016-06-24 Martin Pietsch Add support for VirtualBox 5 2016-06-23 Ján Tomko Rename virNetClient*AddrString Add SASL at the end to make the format obvious. Rename virNetServerClient*AddrString Add SASL at the end to make the format obvious. Add SASL to virNetSocket{Local,Remote}AddrString Rename them to virNetSocket{Local,Remote}AddrStringSASL to make their format more obvious. virNetSocket: rename AddrStr to AddrStrSASL Make it more obvious that these are in the SASL format. virnetsockettest: fix error messages 2016-06-23 Ján Tomko Introduce virNetServerClientRemoteAddrStringURI Use it in virNetServerClientGetInfo to switch back to using the URI-format (separated by ':') instead of the SASL format (separated by ';'). Also use it in the error message reported by virNetServerAddClient. 2016-06-23 Ján Tomko Introduce virNetSocketRemoteAddrStringURI It will return the socket address and port in a URI-like format: [::1]:1234 Add a test case to virnetsockettest. 2016-06-23 Ján Tomko Revert "virnetsocket: Provide socket address format in a more standard form" This partially reverts commit 9b45c9f049a7e9b6c1abfa6988b63b760714e169. It changed the default format of socket address from the one SASL requires, but did not adjust all the callers. It also removed the test coverage for it. Revert most of the changes except the virSocketAddrFormatFull support for URI-formatted strings. This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1345743 while reverting the format used by virt-admin's client-info command from the URI one to the SASL one. https://bugzilla.redhat.com/show_bug.cgi?id=1345743 2016-06-23 Ján Tomko Do not skip files starting with a dot in leases directory '.' and '..' are skipped by virDirRead already. 2016-06-23 Ján Tomko Allow configs to start with a dot This fixes the disappearance of domains and networks starting with a dot. https://bugzilla.redhat.com/show_bug.cgi?id=1333248 2016-06-23 Ján Tomko Do not skip hidden entries when looking for a stable path The device names are unlikely to start with a dot. '.' and '..' are already skipped by virDirRead. Do not ignore hidden files in /sys and /proc The directories we iterate over are unlikely to contain any entries starting with a dot, other than '.' and '..' which is already skipped by virDirRead. Fix comment in virStorageBackendFileSystemRefresh '.' and '..' are now skipped by virDirRead, there's no need to mention them in the comment. Do not check for '.' and '..' after virDirRead It skips those directory entries. Skip '.' and '..' in virDirRead All of the callers either skip these explicitly, skip all entries starting with a dot or match the entry name against stricter patterns. Introduce VIR_DIR_CLOSE Introduce a helper that only calls closedir if DIR* is non-NULL and sets it to NULL afterwards. Do not check the return value of closedir The only possible error is EBADFD. Since we only use the directory stream returned by opendir, this should never happen. Do not save errno in virUSBDeviceSearch The virUSBDeviceFind* callers do not check errno after calling this function. 2016-06-23 Peter Krempa tests: utils: Fail XML file comparison if input file doesn't exist In cases where we expect parse failure of the test input file the testsuite can't differentiate if the parser failed when parsing or when opening the file. Add a call to virFileExists and error out on missing input files. Missing output files are partially expected when regenerating test output. 2016-06-23 Peter Krempa tests: genericxml2xml: Fix test file name Commit b1fc6a7b added a test file but used a different name in the actual test. 2016-06-23 John Ferlan qemu: Add new secret info type Add 'encinfo' to the extended disk structure. This will contain the encryption secret (if present). util: Introduce virSecretLookupFormatSecret Add utility to format the virSecretLookupTypeDefPtr in XML util: Move and rename virStorageAuthDefParseSecret Move to virsecret.c and rename to virSecretLookupParseSecret. Also convert to usage xmlNodePtr and virXMLPropString rather than virXPathString. 2016-06-23 John Ferlan secret: Move virStorageSecretType and rename Move the enum into a new src/util/virsecret.h, rename it to be virSecretLookupType. Add a src/util/virsecret.h in order to perform a couple of simple operations on the secret XML and virSecretLookupTypeDef for clearing and copying. This includes quite a bit of collateral damage, but the goal is to remove the "virStorage*" and replace with the virSecretLookupType so that it's easier to to add new lookups that aren't necessarily storage pool related. 2016-06-23 Cole Robinson storage: Remove redundant refreshPool check Every driver provides a refreshPool impl, and many other critical places in the code unconditionally call it without checking if it exists, so this check is pointless qemu: command: Error on accel2d qemu doesn't have any accel2d support wired up. Explicitly error if a user tries it out, or typos the accel3d option qemu: command: Error on accel3d with non-virtio We should be raising an error if accel3d is present for any non-virtio video as well, incase someone tries it for say 'qxl' 2016-06-23 Nikolay Shirokovskiy qemu: make monitor command API available during async jobs One can not issue monitor commands manually during async calls thru designated API while this could be useful for testing/debugging purposes. qemuDomainQemuMonitorCommand uses job of type QEMU_JOB_MODIFY and any async call disable parallel execution of this type of job. The only state that is changed is taint variable. AFAIU the only place we can mess is resetting taint flag in qemuProcessStop routine under some async job. But this can not happen thanx to both virDomainObjIsActive check in qemuDomainQemuMonitorCommand and resetting active status in qemuProcessStop before taint flag. Change job type to QEMU_JOB_QUERY and thus make the API call available for most of async jobs. 2016-06-23 Ján Tomko prohibit-duplicate-header: print file name and line This way :make syntax-check in ViM will point you at the offending line. syntax-check: rewrite prohibit-duplicate-header in perl Invoke the script only once instead of once for every file. 2016-06-23 Peter Krempa Allow virDomain(SG)etGuestVcpus on read-write connection only Guest agent interaction is considered privileged. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1349272 2016-06-23 Ján Tomko Do not ignore perl scripts in build-aux Also remove the duplicate build-aux entry from .gitignore. Add newDomain parameter to qemuDomainAssignAddresses Pass 'true' if we are not dealing with a migration. Add a USB hub to controller order test The test has too many USB devices. 2016-06-23 Cole Robinson conf: Remove dead console compat formatting This code was attempting to handle some implicit XML formatting for manually assembled DomainDef, since previously the console<->serial compat copying was only done at XML parse time. Nowadays it's done via virDomainDefPostParse -> virDomainDefAddConsoleCompat, which all manual DomainDef builders already call, so we can drop this workaround. 2016-06-22 Jim Fehlig libxl: use serial device for console when targetType is serial When domXML contains only and no corresponding , the console is "stolen" [1] and used as the first device. When this "stolen" console is accessed from the libxl driver (in libxlConsoleCallback and libxlDomainOpenConsole), check if the targetType is VIR_DOMAIN_CHR_CONSOLE_TARGET_TYPE_SERIAL, and use the "stolen" device in def->serials[0] instead. Prior to this change, creating a domain with input XML containing only a device and subsequently attempting to access its console with 'virsh console' would fail error: internal error: character device is not using a PTY [1] See comments associated with virDomainDefAddConsoleCompat() in $LIBVIRT-SRC/src/conf/domain_conf.c: 2016-06-22 Ján Tomko Use for instead of code duplication when parsing USB port We are done if the string ends and move to another nesting level if we find a dot. Split out USB port parsing Make rewriting it easier. 2016-06-22 Ján Tomko Add a test for long USB port paths We support up to four levels of nested USB devices in the guest. Add a test for a domain using all four and a negative test for a domain using five. 2016-06-22 Jiri Denemark Report auto convergence throttle rate in migration stats qemu: Implement auto convergence migration parameters qemu: Add support for cpu throttling parameters qemu: Introduce qemuMigrationSetParams Several places in the code update qemuMonitorMigrationParams structure and qemuMigrationSetParams is then used to set them all at once. Add auto convergence migration parameters They can be used to tune auto-convergence algorithm (which is enabled with VIR_MIGRATE_AUTO_CONVERGE). test: Rework qemuMonitorJSONGetMigrationParams test qemu: Rework qemuMonitorJSONSetMigrationParams qemu: Rework qemuMonitorJSONGetMigrationParams We should not require any parameters to be present. After all we have the *_set bools to express that some parameters were not set. qemu: Rename qemuMonitorMigrationCompression qemuMonitorMigrationParams is a better name for a structure which contains various migration parameters. While doing that, we should use full names for individual parameters. qemu: Decouple migration parameters from compression settings Compression parameters are not the only migration parameters. qemu: Make qemuMonitorSetMigrationCompression saner Checking whether the function has anything to do is better done in the function rather then requiring callers to do that. conf: Simplify conditions in CPU parser/formatter qemucapsprobe: Don't put empty line at EOF 2016-06-22 Jiri Denemark conf: Remove redundant free in virCPUDefFree vendor_id is (and always was) already freed in virCPUDefFreeModel. This effectively reverts commit fb49ffc3. 2016-06-22 Jiri Denemark qemu: Fix reference leak in qemuDomainDefPostParse The function gets a reference on virQEMUDriverConfig which needs to be released before returning. qemu: Hide virQEMUCapsNewForBinary Since virQEMUCapsNewForBinaryInternal was introduced, virQEMUCapsNewForBinary is no longer used outside qemu_capabilities.c. 2016-06-22 Peter Krempa qemu: Implement virDomainSetGuestVcpus Allow modification of specific vCPU states via the guest agent. qemu: Implement virDomainGetGuestVcpus Allow gathering available vcpu ids, their state and offlinability via the qemu guest agent. The maximum id was chosen arbitrarily and ought to be enough for everybody. 2016-06-22 Peter Krempa qemu: agent: Make setting of vcpus more robust Documentation for the "guest-set-vcpus" command describes a proper algorithm how to set vcpus. This patch makes the following changes: - state of cpus that has not changed is not updated - if the command was partially successful the command is re-tried with the rest of the arguments to get a proper error message - code is more robust against malicious guest agent - fix testsuite to the new semantics 2016-06-22 Peter Krempa virsh: Add command 'guestvcpus' implementing virDomain(GS)etGuestVcpus Add a straightforward implementation for using the new APIs. 2016-06-22 Peter Krempa lib: Add API to set individual vcpu usage in the guest via guest agent To allow finer-grained control of vcpu state using guest agent this API can be used to individually set the state of the vCPU. This will allow to better control NUMA enabled guests and/or test various vCPU configurations. 2016-06-22 Peter Krempa lib: Add API to query guest vcpu info using guest agent Add a rather universal API implemented via typed params that will allow to query the guest agent for the state and possibly other aspects of guest vcpus. rpcgen: Add support for generating funcs returning alloc'd typed params Since it's rather tedious to write the dispatchers for functions that return an array of typed parameters (which are rather common) let's add some rpcgen code to generate them. 2016-06-21 John Ferlan tests: Adjust tests for encrypted storage Make them work again... The xml2xml had been working, but the xml2argv were not working. Making the xml2argv work required a few adjustments to the xml to update to more recent times. storage: Use virSecretGetSecretString Rather than inline code secret lookup for rbd/iscsi, use the common function. storage: Create helper to set options for CreateQemuImg code Create a helper virStorageBackendCreateQemuImgSetOptions to set either the qemu-img -o options or the previous mechanism using -F storage: Create helper to set backing for CreateQemuImg code Create a helper virStorageBackendCreateQemuImgSetBacking to perform the backing store set 2016-06-21 John Ferlan storage: Adjust qemu-img switches check Since we support QEMU 0.12 and later, checking for support of specific flags added prior to that isn't necessary. Thus start with the base of having the "-o options" available for the qemu-img create option and then determine whether we have the compat option for qcow2 files (which would be necessary up through qemu 2.0 where the default changes to compat 0.11). Adjust test to no long check for NONE and FLAG options as well was removing results of tests that would use that option. 2016-06-21 Ján Tomko Do not call postParse with ABI_UPDATE when parsing cmdline So far this is only useful for recalculating NUMA memory size, which this function cannot parse. This will let us generate USB addresses based on this flag. 2016-06-21 Ján Tomko Fix USB port in input-usbmouse test The default USB controller only has two ports. test-wrap-argv: add --check parameter This script can already operate on a list of files. Add a --check parameter to check if multiple files are wrapped correctly with a single invocation of the script. test-wrap-argv: add --in-place parameter If --in-place is supplied as the first argument to the script, replace the file in-place instead of printing to stdout. test-wrap-argv: hold a copy of the original file in an array This will be useful to check if the file is wrapped already. test-wrap-argv: return a string in rewrap_line Leave the printing up to &rewrap. 2016-06-21 Ján Tomko test-wrap-argv: use map and join instead of a for cycle We have a list of parameters in @args, that need to be rewrapped and separated by a space and escaped newline: " \\\n", with the exception of the last one, which only needs a newline. Instead of a for cycle, rewrap the individual arguments using map, and interleave them with escaped newlines by using join. 2016-06-21 Ján Tomko test-wrap-argv: return a string in rewrap_arg Do not print anything, let the caller take care of it. test-wrap-argv: split out rewrap_arg Split out the code wrapping the single argument. 2016-06-21 Ján Tomko test-wrap-argv: split out rewrap_line Shorten the rewrap subroutine by splitting out the code dealing with a single line. Also remove $file from the warning. 2016-06-21 Ján Tomko vsh: remove namespace poisoning We already have a syntax-check to prohibit direct use of these allocation functions. 2016-06-21 Ján Tomko vbox: remove duplicate macros There is a definiton of VIR_FROM_THIS just two lines above. The rest is defined in vbox_common.h. 2016-06-21 Ján Tomko Remove unused SOL_NETLINK macro Introduced by commit d575679, unused at the time. 2016-06-21 Laine Stump conf: limit chassisNr, and busNr to a minimum value of 1, not 0 In the case of chassisNr (used to set chassis_nr of a pci-bridge controller), 0 is reserved for / used by the pci[e]-root bus. In the base of busNr, a value of 0 would mean that the root bus had no places available to plug in new buses, including the pxb itself (the documentation I wrote for pxb even noted the limit of busNr as 1.254). NB: oddly, the "chassis" attribute, which is used for pcie-root-port and pcie-switch-downstream-port *can* be set to 0, since it's the combination of {chassis, slot} that needs to be unique, not chassis by itself (and slot 0 of pcie-root is reserved, while pcie-*-port can use *only* slot 0). This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1342962 2016-06-21 Andrea Bolognani qemu: Fix typo pci-extender-bus -> pci-expander-bus 2016-06-21 Jiri Denemark virsh migrate: Fix positional parameters Thanks to our smart option parser which automatically assigns positional parameters the following (previously working) command fails: virsh migrate test qemu+ssh://1.2.3.4/system tcp://1.2.3.4/ error: invalid argument: Unsupported compression method 'tcp://1.2.3.4/' We need to make sure new options are added at the end of the list rather than where they logically belong. Reported by Brian Rak. 2016-06-20 Ján Tomko Mark virsh-optparse as expensive 2016-06-20 Ján Tomko Drop virrandomtest This test only checks if mocking of virRandomBytes works correctly. Drop it to avoid infinite recursion by testing the test suite. 2016-06-20 Ján Tomko Remove virsh-synopsis This tests checks that the first word after SYNOPSIS in virsh help ${command} output is ${command}. This was only good to check that the command option structures are valid, which is now served by 'virsh self-test'. 2016-06-20 Ján Tomko Introduce virsh self-test A new hidden command for virsh that will iterate over all command groups and commands and print help for every single one. This involves running vshCmddefOptParse so we can get an error if one of the command's option structure is invalid. 2016-06-20 Ján Tomko tests: mock gnutls_dh_params_generate2 This function generates some big random numbers. Cache the result and supply it to any subsequent generate2 calls. 2016-06-20 Ján Tomko Remove virsh-all Since e8ac4a7 this test wastes some CPU cycles by blindly trying to run almost every virsh command, blindly throwing away the output and the return value and returning success if 'virsh help' successfully returned at least one command. Drop it completely. 2016-06-20 Peter Krempa util: Make failure to get supplementary group list for a uid non-fatal Since introduction of the DAC security driver we've documented that seclabels with a leading + can be used with numerical uid. This would not work though with the rest of libvirt if the uid was not actually used in the system as we'd fail when trying to get a list of supplementary groups for the given uid. Since a uid without entry in /etc/passwd (or other user database) will not have any supplementary groups we can treat the failure to obtain them as such. This patch modifies virGetGroupList to not report the error for missing users and makes it return an empty list or just the group specified in @gid. All callers will grant less permissions to a user in case of failure of this function and thus this change is safe. 2016-06-20 Peter Krempa util: Add option not to report errors in virGetUserEnt In some cases it will be necessary to ignore errors reported from this function. This allows suppressing them to avoid spamming logs. tools: virt-login-shell: Fix cut'n'paste mistake in error message Whine about 'allowed_users' having wrong format rather than 'shell' tools: virt-login-shell: Fix group list bounds checking The list certainly isn't zero terminated and it would disallow usage of group 'root'. Pass in the array size and match against it. conf: Fix label name in virDomainGraphicsListensParseXML Use 'cleanup' since it's also used on success. 2016-06-20 Peter Krempa conf: Fix memory leak in graphics XML parser When loading status XMLs with following graphics definition: libvirtd would leak a few bytes: 10 bytes in 1 blocks are definitely lost in loss record 71 of 1,127 at 0x4C2C000: malloc (vg_replace_malloc.c:299) by 0x6789298: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4) by 0x552AB0A: virXMLPropString (virxml.c:479) by 0x5539536: virDomainGraphicsListensParseXML (domain_conf.c:11171) by 0x553DD5E: virDomainGraphicsDefParseXMLSpice (domain_conf.c:11414) by 0x553DD5E: virDomainGraphicsDefParseXML (domain_conf.c:11749) by 0x5566061: virDomainDefParseXML (domain_conf.c:16939) by 0x556953F: virDomainObjParseXML (domain_conf.c:17348) by 0x556953F: virDomainObjParseNode (domain_conf.c:17513) by 0x5569902: virDomainObjParseFile (domain_conf.c:17532) by 0x5571E02: virDomainObjListLoadStatus (virdomainobjlist.c:514) by 0x5571E02: virDomainObjListLoadAllConfigs (virdomainobjlist.c:596) by 0x26E0BDC8: qemuStateInitialize (qemu_driver.c:911) by 0x55B1FDB: virStateInitialize (libvirt.c:770) by 0x122039: daemonRunStateInit (libvirtd.c:960) 2016-06-20 Chen Hanxiao util: fix a typo s/succcess/success 2016-06-20 Jaroslav Suchanek docs: virsh: Added note for the dump command Crash dump in a old kvmdump format is being obsolete and cannot be loaded and processed by crash utility since its version 6.1.0. A --memory-only option is required in order to produce valid ELF file which can be later processed by the crash utility. A new note is added to the dump command description. 2016-06-20 Ján Tomko Allow disjunct ranges in VIR_TEST_RANGE Use virBitmapParseUnlimited to parse the env variable. Introduce virBitmapParseUnlimited For parsing a bitmap of an unknown size. Remove separator argument from virBitmapParse Most the callers pass 0 in one form or another, including vircapstest which used VIR_ARCH_NONE. 2016-06-20 Ján Tomko Introduce virBitmapParseSeparator This will be used for the caller that needs to specify a separator. Currently identical to virBitmapParse. Also change one test case to use the new function. 2016-06-20 Ján Tomko Do not return number of set bits in virBitmapParse This is only used by one caller. 2016-06-20 Andrea Bolognani qemu: Don't use legacy USB for aarch64 mach-virt guests The '-usb' option doesn't have any effect for aarch64 mach-virt guests, so the fact that it's currently enabled by default is not really causing any issue. However, that might change in the future (although unlikely), and having it as part of the QEMU command line can cause confusion to someone looking through the process list. Avoid it completely, like it's already happening for q35. 2016-06-19 Tomasz Flendrich qemu_hotplug: Use a helper variable consistently 2016-06-18 Jovanka Gulicoska virsh: Introduce pool-event command Similar to 'event' and 'net-event', this prints info about incoming storage pool events. 2016-06-17 Jim Fehlig libxl: add USB to hostdev domcapabilities Commit 2a58ed0b added support for creating guests with USB hostdevs. Commit fc21d10 later added support for hotplut of USB hostdevs. Advertise support for USB hostdevs in the domcapabilities. In addition add the appropriate caps for USB support on domaincapstest when libvirt is built on a Xen with LIBXL_HAVE_PVUSB. Otherwise domaincapstest would fail i.e. testing the wrong domain capabilities. 2016-06-17 Ján Tomko qemu: restore non-pci hostdev labels after detach Commit 409de00 changed the logic to only match PCI devices while moving this before Remove*HostDevice calls. https://bugzilla.redhat.com/show_bug.cgi?id=1342874 2016-06-17 Ján Tomko Remove stray space in cmdHelp 2016-06-17 Andrea Bolognani qemu: Permit PCI-free aarch64 mach-virt guests There has been some progress lately in enabling virtio-pci on aarch64 guests; however, guest OS support is still spotty at best, so most guests are going to be using virtio-mmio instead. Currently, mach-virt guests are closely modeled after q35 guests, and that includes always adding a dmi-to-pci-bridge that's just impossible to get rid of. While that's acceptable (if suboptimal) for q35, where you will always need some kind of PCI device anyway, mach-virt guests should be allowed to avoid it. 2016-06-17 Michal Privoznik qemuDomainDetachDeviceConfig: Allow cold unplug of redirdevs This is fairly simple. We lookup the device in the array of devices and remove it. No magic. qemuDomainAttachDeviceConfig: Allow redirdev coldplug This is really simple, we just need to append the device into the domain def and that's it. virDomainDeviceInfoIterateInternal: Iterate through redirdevs too This is going to be important later when we received DEVICE_DELETED event on the qemu monitor. If we do, virDomainDefFindDevice() is called to find the device for given device alias in the virDomainDef tree. When we enable removal for redirdevs we need to include them in the lookup process too. virDomainRedirdevDef: Introduce find & remove routines Basically, there are just two functions introduced here: virDomainRedirdevDefFind which looks up given redirdev in domain definition, and virDomainRedirdevDefRemove which removes the device at given index in the array of devices. domain_conf: Validate redirdev after parsing There's currently just one limitation: redirdevs that want to go on USB bus require a USB controller, surprisingly. At the same time, since I'm using virDomainDefHasUSB() in this new validator function, it has to be moved a few lines up and also its header needed to be changed a bit: it is now taking a const pointer to domain def since it's not changing anything in there. 2016-06-17 Ján Tomko configure: remove definition of HAVE_GLIBC_RPCGEN Unused since commit fb1e8d9 in May 2011. configure: error out when asked for mpath on non-Linux 2016-06-17 Ján Tomko configure: define preprocessor macros for SCSI and MPATH This fixes building these backends when explicitly enabled on the command line. Exposed by commit a659559 which started adding --with-storage-mpath in the spec file. https://bugzilla.redhat.com/show_bug.cgi?id=1346724 2016-06-17 Andrea Bolognani qemu: Fix alignment in virDomainDefAddController() call 2016-06-17 Peter Krempa tests: schema: Remove useless perf schema data We have a test case that excercises the parser and formatter now which takes part in schema checking so remove the schema-only test. docs: virsh: Add minimal documentation for 'mbmt' and 'mbml' perf events 2016-06-17 Jiri Denemark cpu_x86: Use signature in CPU detection code Our current detection code uses just the number of CPU features which need to be added/removed from the CPU model to fully describe the CPUID data. The smallest number wins. But this may sometimes generate wrong results as one can see from the fixed test cases. This patch modifies the algorithm to prefer the CPU model with matching signature even if this model results in a longer list of additional features. 2016-06-17 Jiri Denemark cpu: Add Skylake-Client x86 CPU model The CPU model was implemented in QEMU by commit f6f949e929. The change to i7-5600U is wrong since it's a 5th generation CPU, i.e., Broadwell rather than Skylake, but that's just the result of our CPU detection code (which is fixed by the following commit). 2016-06-17 Martin Kletzander spec: Make driver-qemu require driver-storage Without that we might get similar messages in the log: error : virDriverLoadModule:73 : failed to load module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so: undefined symbol: virStorageFileCreate 2016-06-17 Peter Krempa conf: Rename virDomainDefGetMemoryActual to virDomainDefGetMemoryTotal 2016-06-17 Peter Krempa conf: Remove pre-calculation of initial memory size While we need to know the difference between the total memory stored in and the actual size not included in the possible memory modules we can't pre-calculate it reliably. This is due to the fact that libvirt's XML is copied via formatting and parsing the XML and the initial memory size can be reliably calculated only when certain conditions are met due to backwards compatibility. This patch removes the storage of 'initial_memory' and fixes the helpers to recalculate the initial memory size all the time from the total memory size. This conversion is possible when we also make sure that memory hotplug accounts properly for the update of the total memory size and thus the helpers for inserting and removing memory devices need to be tweaked too. This fixes a bug where a cold-plug and cold-remove of a memory device would increase the size reported in in the XML by the size of the memory device. This would happen as the persistent definition is copied before attaching the device and this would lead to the loss of data in 'initial_memory'. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1344892 2016-06-17 Peter Krempa conf: Fix perf event parser The parser was totaly broken. Fix it by rewriting it. Add tests so that it doesn't happen. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1346723 2016-06-17 Peter Krempa docs: Add at least some docs and fix schema entry for perf events There was no documentation at all for the XML part. I added at least some. The 2.0.0 introduction date is deliberate as the parser for the XML is broken. The schema file was missing entries for 'mbml' and 'mbmt'. 2016-06-17 Peter Krempa qemu: perf: Don't set state of first event for every other event A bug in the code used the value of the first perf event as state for all the mentioned one rather than extracting individual ones. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1346730 2016-06-16 Jim Fehlig xenconfig: fix conversion of to backendtype When converting domXML to xen xl.cfg, backendtype should not be emitted if is not specified. Moreover, should be converted to backendtype qdisk, similar to handling of in libxlMakeDisk() in libxl_conf.c. Prior to this change, connectDomainXMLToNative would produce incorrect xl.cfg when the input domXML contained domXML: virsh domxml-to-native xen-xl domXML disk = [ "format=raw,vdev=xvda,access=rw,backendtype=target=/image/file/path" ] xl create xl.cfg config parsing error in disk specification: unknown value for backendtype: near `target=/image/file/path' in `format=raw,vdev=xvda,access=rw,backendtype=target=/image/file/path' 2016-06-16 Laine Stump util: fix missing broadcast address in bridge and tap device IP addresses Commit b3d069872ce53eb added peer address setting to the low level virNetDevSetIPAddress() function, but ended up causing a segfault in cases where the caller passed NULL for peer address. Commit a3510e33d33e52c fixed the segfault, but managed to cause us to skip setting the broadcast address when setting an interface's IP address. The result is that the broadcast address is 0.0.0.0 for all libvirt-created bridges (and interfaces in lxc containers with IP addresses set by libvirt). This was reported on the mailing list: https://www.redhat.com/archives/libvir-list/2016-June/msg00027.html but I was too busy to investigate at the time. I found it by accident today while refactoring virNetDevSetIPAddress(). Since this regression is present in the 1.3.5 release, I'm sending the bugfix as a separate patch from my larger refactoring patchset. 2016-06-16 Laine Stump qemu: don't add pci-bridge to Q35/arm domains unless it's needed Until now, a Q35 domain (or arm/virt, or any other domain that has a pcie-root bus) would always have a pci-bridge added, so that there would be a hotpluggable standard PCI slot available to plug in any PCI devices that might be added. This patch removes the explicit add, instead relying on the pci-bridge being auto-added during PCI address assignment (it will add a pci-bridge if there are no free slots). This doesn't eliminate the dmi-to-pci-bridge controller that is explicitly added whether or not a standard PCI slot is required (and that is almost never used as anything other than a converter between pcie.0's PCIe slots and standard PCI). That will be done separately. 2016-06-16 Laine Stump qemu: don't be as insistent about adding dmi-to-pci-bridge or pci-bridge Previously there was no way to have a Q35 domain that didn't have these two controllers. This patch skips their creation as long as there are some other kinds of pci controllers at index 1 and 2 (e.g. some pcie-root-port controllers). I'm hoping that soon we won't add them at all, plugging all devices into auto-added pcie-*-port ports instead, but in the meantime this makes it easier to experiment with alternative bus hierarchies. 2016-06-16 Chen Hanxiao util: remove redundant comments 2016-06-16 Jovanka Gulicoska event-test: support storage lifecycle event APIs storage: implement storage lifecycle event APIs Implement storage pool event callbacks for START, STOP, DEFINE, UNDEFINED and REFRESHED in functions when a storage pool is created/started/stopped etc. accordingly remote: implement storage lifecycle event APIs test: implement storage lifecycle event APIs Also includes unittests for storage pool lifecycle events API conf: add storage_event handling Add storage event handling infrastructure to storage_event.[ch], following the network_event.[ch] pattern. 2016-06-16 Jovanka Gulicoska Introduce storage lifecycle event APIs Storage pool lifecycle event API entry points for registering and deregistering storage pool events, as well as types of events associated with storage pools. These entry points will be used for implementing asynchronous lifecycle events. Storage pool API: virConnectStoragePoolEventRegisterAny virConnectStoragePoolEventDeregisterAny virStoragePoolEventLifecycleType which has events STARTED, STOPPED, DEFINED, UNDEFINED, and REFRESHED 2016-06-16 Martin Kletzander qemu: Remove useless block in processWatchdogEvent Best viewed with -w since this is mostly just an indentation patch. qemu: Follow coding style convention qemu: Unify automatic coredump filenames Just create a helper for it and use it. 2016-06-16 John Ferlan qemu: Add cfg pointer to various command line helpers Soon at least one of them will need to grab something out of the qemu config structure qemu: Make qemuBuildShmemBackendStr private It's not used externally anywhere else 2016-06-16 Martin Kletzander qemu: Shorten domain name for watchdog coredump Similarly to commit d294f6b0dff7, if the name is long enough, the filename can be longer than filesystem's limit. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1334237 2016-06-16 Cole Robinson qemu: migration: use consistent error message The other two DomainHasBlockJob usage error messages don't contain 'an', so unify things to save translators some effort. Dropping the 'an' is closer to the sentence structure in the errors from qemuDomainDiskBlockJobIsActive as well qemu: More usage of qemuDomainDiskBlockJobIsActive qemuDomainDiskBlockJobIsActive already checks if a disk has a blockjob, and if so, raises an error 2016-06-16 sannyshao virsh: Fix help string for net-dumpxml Follow other examples (dumpxml, iface-dumpxml, and pool-dumpxml) for the --inactive .help string 2016-06-16 John Ferlan util: Fix broken syntax-check Commit id '743db933' broke at least one syntax check rule regard open/close braces and perhaps more with spacing. Just remove the braces 2016-06-16 yuelongguang add default mapping of credentials to machine In the auth config file, it is currently required to have an entry for each hostname to connect to, eg [auth-libvirt-prod1.example.com] credentials=prod This is inconvenient when there are large numbers of machines all with the same credentials. Add support for a default entry: [auth-default] credentials=prod 2016-06-16 Ján Tomko syntax-check: drop prohibit_undesirable_word_seq This checks forbids using "can not" and checks the placement of some texinfo tags. Drop it since we do not use texinfo and the check takes almost twice as much as the rest of the checks. 2016-06-15 Andrea Bolognani maint: Switch to xz compressed PAX release archives This allows us to produce releases that are roughly a third in size, have no limitation on path length, and are still readable by all supported platforms. 2016-06-15 Nikolay Shirokovskiy vz: get rid of unused home state variable in private domain obj 2016-06-15 Ján Tomko check-spacing: fix error message The check for whitespace before comma or semicolon referred to comma as colon. check-spacing: use non-capturing groups check-spacing: remove virAssertCmpInt exception The macro is now called testAssertEq and no longer takes an operator as its argument. 2016-06-15 Ján Tomko check-spacing: simplify keyword spacing check We do not need a separate check forbidding whitespace after the opening parenthesis after a keyword - we forbid it after all of them. The only allowed whitespace after an opening parenthesis is a newline, tune the regex to reflect that. 2016-06-15 Ján Tomko check-spacing: rewrite regex for checking the closing parenthesis Instead of matching multiple characters before the parenthesis, only check for a single whitespace, which is much less cpu-intensive. This only matches a few dozen of places where they are on an separate line, filter out those with a separate regex. 2016-06-15 Ján Tomko check-spacing: rewrite whitespace check before (semi)colon Instead of matching multiple characters, match any occurrence preceded by a single whitespace and handle the exceptions later. Rename bracket-spacing.pl to check-spacing.pl We test whitespace with lots of other characters now. Rename virAssertCmpInt to testAssertEq Drop the op parameter, we only use equality. Drop the vir prefix since it's only used in the tests. maint: remove whitespace from closing parentheses To allow tightening syntax check. vbox: reformat multi-line error reports Put the comma on the first line. 2016-06-15 Ján Tomko cfg.mk: use a single regex for all non-reentrant functions The prohibit_nonreentrant syntax-check rule spawns a new shell for every non-reentrant function we know, to make it easier to mention the function name in the error message, with the _r appended. Since the line with the offending function is already printed and some of the functions on our list do not have a _r counterpart, compile them into one big regex and use a more generic error message to save time. 2016-06-15 Ján Tomko docs: document rng backend path restrictions of older libvirt Commit 67f2b72 removed the path restrictions for rng backend, along with the documentation of the restriction. Restore the documentation for users still using older libvirt. https://bugzilla.redhat.com/show_bug.cgi?id=1074464 2016-06-15 Michal Privoznik virHostCPUGetInfo: Fix build on non-Unix like systems This function is plenty of ifdefs providing implementations for Linux, *BSD and OS-X. However, if we are being build for any other architecture, all that's left behind by preprocessor is just a error reporting call and return of -1. In that case, passed arguments are unused: ../../src/util/virhostcpu.c: In function 'virHostCPUGetInfo': ../../src/util/virhostcpu.c:966:33: error: unused parameter 'cpus' [-Werror=unused-parameter] unsigned int *cpus, ^~~~ 2016-06-15 Jingjing Shao doc: Fix explanation of S3 and S4 states Explanation of S3 and S4 states was swapped. 2016-06-14 Chunyan Liu xlconfigtest: add test for USB config conversion 2016-06-14 Chunyan Liu Add conversion of domxml USB config to/from xl.cfg xl.cfg: usbdev = [ "hostbus=1,hostaddr=3" ] usb.xml:
2016-06-14 Chunyan Liu libxl: support hotplug USB host device Support hot attach/detach a USB host device to guest. Currently libxl only supports xen PV guest, and only supports specifying USB host device by 'bus number' and 'device number', for example: usb.xml:
#xl attach-device dom usb.xml #xl detach-device dom usb.xml 2016-06-14 Chunyan Liu libxl: support creating guest with USB hostdev Support creating guest with USB host device in config file. Currently libxl only supports xen PV guest, and only supports specifying USB host device by 'bus number' and 'device number', for example:
2016-06-14 John Ferlan qemu: Refactor qemuDomainAttachChrDevice error paths Refactor the error paths for attaching char device (it's about to be more complicated). docs: Clarify chardev protocol Add a slight clarification to usage of "telnet", "telnets", or "tls" as the protocol type value. caps: Add capability for tls-x509-creds Add the capability flag and checks for the qemu object 'tls-creds-x509' docs: Fix syntax-check Commit id '42ff399a' broke syntax-check by not encasing
elements in xxx 2016-06-14 Martin Kletzander Fix renumbering once again I screwed up by accidentally pushing incomplete version of the renumbering commit. This patch just fixes the rest so the tree matches changes in v2. Change 1.3.6 occurrences to 2.0.0 to follow version bump Version was bumped but documentation (and comments) didn't follow the numbering. 2016-06-14 Martin Kletzander qemu: Allow ACPI shutdown only for running domains If the domain is not running, but for example the CPUs are stopped, the ACPI event gets queued and resume of the domain will just shut it off. https://bugzilla.redhat.com/show_bug.cgi?id=1216281 2016-06-14 Martin Kletzander qemu: Obtain job before checking if domain is live Since obtaining a job can wait for another job to finish, the state might change in the meantime. And checking it more than once is pointless. 2016-06-14 Daniel P. Berrange Bump release to 2.0.0 and document release schedule & versioning This bumps the release number of 2.0.0, to reflect the switch to a new time based release versioning scheme. The downloads page is updated to describe our policies for release schedules and release version numbering The stable release docs are changed to reflect the fact that the stable version numbers are now just 3 digits long instead of 4. 2016-06-14 Jiri Denemark cputest: Get rid of the array of test functions 2016-06-14 Martin Kletzander qemu: Add support for zero-detection writes conf: Add support of zero-detection for disks This option allows or disallows detection of zero-writes if it is set to "on" or "off", respectively. It can be also set to "unmap" in which case it will try discarding that part of image based on the value of the "discard" option. Fix build without xen Commit 11567cf66f36 introduced an include which will only work when building with xen (particularly libxl). However, that file is supposed to be includable from anywhere (as with other testutils* files. 2016-06-14 Roman Bogorodskiy virt-host-validate: fix build with clang Building with clang 3.8 triggers the following error: CC virt_host_validate-virt-host-validate-qemu.o virt-host-validate-qemu.c:36:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers] char *kvmhint = _("Check that CPU and firmware supports virtualization " ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ virt-host-validate-qemu.c:46:17: error: assigning to 'char *' from 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers] kvmhint = _("Check that the 'kvm-intel' or 'kvm-amd' modules are " ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2 errors generated. Fix by using 'const' for kvmhint declaration. 2016-06-13 Jim Fehlig libxl: Add support for ovmf firmware Populate libxl_domain_build_info struct with bios and firmware info from virDomainLoaderDef. Note: Currently libxl only allows specifying the type of BIOS. For type LIBXL_BIOS_TYPE_OVMF, the firmware path is configured when building Xen using '--with-system-ovmf='. If not specified, LIBXL_FIRMWARE_DIR/ovmf.bin is used. In the future, Xen will support a user-specified firmware path. See http://lists.xenproject.org/archives/html/xen-devel/2016-03/msg01628.html Once that work is merged into xen.git, the libvirt libxl driver will be able to honor a user-specified path. In the meantime use the implicit path, which is tolerable since it is advertised in domcapabilities. 2016-06-13 Jim Fehlig xenconfig: support bios=ovmf xl.cfg Add support to xenconfig for conversion of xl.cfg(5) bios config to/from libvirt domXml config. SeaBIOS is the default for HVM guests using upstream QEMU. ROMBIOS is the default when using the old qemu-dm. This patch allows specifying OVMF as an alternate firmware. Example xl.cfg: bios = "ovmf" Example domXML: ... /usr/lib/xen/boot/ovmf.bin Note that currently Xen does not support a separate nvram for non-volatile variables. 2016-06-13 Jim Fehlig libxl: implement connectGetDomainCapabilities Add domain capabilities for PV and HVM domains. libxl: introduce libxl_capabilities.{ch} Move capabilities code out of libxl_conf.{ch} and into new libxl_capabilities.{ch} files. libxl: add default firmwares to driver config object Prefer firmwares specified via --with-loader-nvram configure option. If none are specified, use the Xen-provided default firmwares found in LIBXL_FIRMWARE_DIR. 2016-06-13 Jim Fehlig driver config: Introduce virFirmware object The virQEMUDriverConfig object contains lists of loader:nvram pairs to advertise firmwares supported by by the driver, and qemu_conf.c contains code to populate the lists, all of which is useful for other drivers too. To avoid code duplication, introduce a virFirmware object to encapsulate firmware details and switch the qemu driver to use it. 2016-06-13 Wang Yufei libxl: fix vm lock overwritten bug In libxl driver we do virObjectRef in libxlDomainObjBeginJob, If virCondWaitUntil failed, it goes to error, do virObjectUnref, There's a chance that someone undefine the vm at the same time, and refs unref to zero, vm is freed in libxlDomainObjBeginJob. But the vm outside function is not Null, we do virObjectUnlock(vm). That's how we overwrite the vm memory after it's freed. I fix it. 2016-06-13 Riku Voipio virt-host-validate: improve tests for arm/aarch64 ARM/Aarch64 /proc/cpuinfo has no virtualization related flags. Refactor the Qemu/KVM test a bit: 1) run the "for hardware virtualization" test only on plaforms with known cpuinfo flags (x86, s390) 2) test for /dev/kvm also on platforms where no cpu flags are set Finally Add a more generic error hint message for non-x86 plaforms when /dev/kvm is missing. 2016-06-13 Wei Liu tests: fix CPUID detection tests compilation failure In 3704b9003 ("tests: Add CPU detection tests"), a macro called DO_TEST_CPUID_JSON is added. But it took only two arguments when QEMU or YAJL is not set. Fix it by adding a third argument. Shouldn't have any effect because that macro compiles to nothing. 2016-06-13 Roman Bogorodskiy virt-login-shell: mark as Linux only Currently, virt-login-shell is not allowed to build on Windows. However, as it's designed around LXC, it does not make sense to build it on anything but Linux, so make the check stricter and allow to enable it on Linux only. 2016-06-12 Guido Günther apparmor: Don't scrub environment of virtlogd process otherwise we drop variables like XDG_RUNTIME_DIR with qemu:///session and libvirtd faild to find virtlogd's socket. 2016-06-12 Roman Bogorodskiy bhyve: fix bhyvexml2arg test Don't use duplicating target dev names. bhyve: add missing virhost(cpu|mem).h headers hostcpu: fix build on FreeBSD * Fix misspelt function name: s/virHostCPUGetStatsFreebsd/virHostCPUGetStatsFreeBSD/ * Mark the first argument to virHostCPUGetInfo with ATTRIBUTE_UNUSED as it's not actually used on non-Linux 2016-06-12 Roman Bogorodskiy nodeinfo: fix build on non-Linux SYSFS_SYSTEM_PATH is only defined for Linux, however it's used outside of #ifdef __linux__ code, e.g. as the first argument to nodeCapsInitNUMAFake(). But as this argument's value is used on Linux only, it's safe to define SYSFS_SYSTEM_PATH to "fake" to get things built on FreeBSD. 2016-06-11 Maxim Nestratov vz: fix crash when parsing unexpected disk configuration As it turned out PrlVmDev_GetStackIndex can return negative values without reporting an error, which is incorrect but nevertheless. After that we feed this negative index to virIndexToDiskName, which in turn returns NULL and we set it to virDomainDiskDef.dst. Using virDiskNameToBusDeviceIndex with a virDomainDiskDef structure which has NULL dst field crashes. Fix this by returning an error in prlsdkGetDiskId in such cases. 2016-06-11 Mikhail Feoktistov vz: implementation of domainSetUserPassword callback 2016-06-11 Maxim Nestratov vz: return correct result for unimplemented ChangeState actions Map PRL_ERR_UNIMPLEMENTED to VIR_ERR_OPERATION_INVALID vz: remove unused macro logPrlEventError 2016-06-11 Nikolay Shirokovskiy vz: keep subscription to performance events thru domain lifetime The approach of subscribing on first stat API call and then waiting for receiving of performance event from sdk to process the call originates in times when every vz libvirt connections spawns its own sdk connection. Thus without this waiting virsh stat call would return empty stats. Now with single sdk connection this scheme is unnecessary complicated. This patch subscribes to performance events on first domain appearence and unsubscribe on its removing. 2016-06-11 Nikolay Shirokovskiy vz: use consistent naming for different domain object in vz_driver.c Naming scheme is next: virDomainPtr domain; virDomainObjPtr dom; 2016-06-10 Pavel Hrdina domain_conf: silence gcc warnings ../../src/conf/domain_conf.c:10949: error: declaration of 'socket' shadows a global declaration [-Wshadow] ../../src/conf/domain_conf.c:24373: error: declaration of 'listen' shadows a global declaration [-Wshadow] 2016-06-10 Maxim Nestratov vz: fixed build by including necessary headers After eaf18f4c2 some functions changed their homes Pushed under build breaking rule 2016-06-10 Michal Privoznik qemuMonitorJSONAttachCharDev: Teach spicevmc https://bugzilla.redhat.com/show_bug.cgi?id=1298070 We have the code for attaching redirdevs for ages now. Unfortunately, our monitor code that handles talking to the qemu process was missing a little piece of code that actually enabled the feature. BTW: it really is called "type" on the monitor, even though it's called "name" on the cmd line. Don't ask. 2016-06-10 Michal Privoznik Export virDomainRedirdevDefFree In the 162efa1a commit the function was introduced, but the commit forgot to update livirt_private.syms accordingly. 2016-06-10 Daniel P. Berrange virt-login-shell: add ability to join the container cgroups Prior to joining the namespaces of the container, move the process into the containers' cgroups, so that the shell that is subsequently launched is under the container resource constraints. virt-login-shell: add ability to auto-detect shell from container Currently the shell must be looked up from the config setting in /etc/libvirt/virt-login-shell.conf. This is inflexible if there are containers where different users need different shells. Add add a new 'auto-shell' config parameter which instructs us to query the containers' /etc/passwd for the shell to be exec'd. 2016-06-10 Daniel P. Berrange virt-login-shell: fully reset container environment The virt-login-shell environment will be initialized with an arbitrary number of environment variables determined by the SSH daemon and PAM configuration. Most of these are not relevant inside the container, and at best they are noise and at worst they'll break apps. For example if XDG_RUNTIME_DIR is leaked to the container, it'll break any apps using it, since the directory it points to is only visible to the host OS filesystem, not the container FS. Use clearenv() to blank out everything and then set known good values for PATH, SHELL, USER, LOGNAME HOME and TERM. Everything else is left up to the login shell to initialize. 2016-06-10 Daniel P. Berrange virt-login-shell: avoid loosing error during cleanup The virDomainFree / virConnectClose methods will reset the last error handle, so we must save the error during cleanup virt-login-shell: allow shell to be a simple string argument Currently the shell config file parameter must be a list giving the shell path and args. Allow it to be a plain string argument as well. 2016-06-10 Daniel P. Berrange virt-login-shell: change way we request a login shell Currently we request a login shell by passing the -l argument to the shell. This is either hardcoded, or required to be specified by the user in the virt-login-shell.conf file. The standard way for login programs to request a shell run as a login shell is to modify the argv passed to execve() so that argv[0] contains the relative shell filename prefixed with a zero. eg instead of doing const char **shellargs = ["/bin/bash", "-l", NULL]; execve(shellargs[0], shellargs, env); We should be doing const char **shellargs = ["-bash", NULL]; execve("/bin/bash", shellargs, env); 2016-06-10 Daniel P. Berrange virt-login-shell: honour the -c option to launch commands The virt-login-shell program is supposed to look like a regular shell to clients. Login services like sshd expect the shell to accept a '-c cmdstring' argument to specify a command to launch instead of presenting an interactive prompt. We can implement this by simply passing the '-c cmdstring' data straight through to the real shell we use. This does not open any security holes, since the command is not run until we're inside the container namespaces. This allows scp to work for users with virt-login-shell. 2016-06-10 Daniel P. Berrange virsh: make lxc-enter-namespace also join the cgroups Extend the lxc-enter-namespace command so that it joins the containers' cgroups before starting new namespaces. This ensures that the commands run have the normal resource limits applied libvirt-lxc: add virDomainLxcEnterCGroup API Add the virDomainLxcEnterCGroup API to the libvirt-lxc.so file. This method moves the calling process into the cgroups associated with the container. 2016-06-09 Daniel P. Berrange util: add function for looking up the user shell Add a virGetUserShell wrapper around virGetUserEnt, that returns the shell field. nodeinfo: move host memory APIs out into virhostmem file Move all APIs with a virHostMEM name prefix out into new util/virhostmem.h & util/virhostmem.c files nodeinfo: move host CPU APIs out into virhostcpu.c file Move all APIs with a virHostCPU name prefix out into new util/virhostcpu.h & util/virhostcpu.c files nodeinfo: rename all CPU APIs to have a virHostCPU prefix In preparation for moving all the CPU related APIs out of the nodeinfo file, give them a virHostCPU name prefix. nodeinfo: rename all memory APIs to have a virHostMem prefix In preparation for moving all the memory related APIs out of the nodeinfo file, give them a virHostMem name prefix. nodeinfo: split CPU info retrieval out of nodeGetInfo Instead of having platform specific code in nodeGetInfo to fetch CPU topology, split it all out into a new method nodeGetCPUInfo. nodeinfo: remove FreeBSD specific code for getting memory The GNULIB physmem module already provides support for the FreeBSD platform, so there's no reason to re-implement FreeBSD portability code in libvirt. If there are bugs in the GNULIB code, we should fix GNULIB rather than workaround it in libvirt. nodeinfo: make nodeGetInfo() call nodeGetMemory for memory size The nodeGetInfo() method currently has its own code for getting memory size in KB, that basically just re-invents what nodeGetMemory already does. Remove it and just call nodeGetMemory, converting its result from bytes to KB, allowing removal of more platform specific conditional code. 2016-06-09 Daniel P. Berrange nodeinfo: remove sysfs_prefix from all methods Nearly all the methods in the nodeinfo file are given a 'const char *sysfs_prefix' parameter to override the default sysfs path (/sys/devices/system). Every single caller passes in NULL for this, except one use in the unit tests. Furthermore this parameter is totally Linux-specific, when the APIs are intended to be cross platform portable. This removes the sysfs_prefix parameter and instead gives a new method linuxNodeInfoSetSysFSSystemPath for use by the test suite. For two of the methods this hardcodes use of the constant SYSFS_SYSTEM_PATH, since the test suite does not need to override the path for thos methods. 2016-06-09 Martin Kletzander qemu: Yet another check for blkdeviotune values If you want to set block device I/O tuning values that end with '_max' and there is nothing else set, libvirt emits an error. In particular: error: internal error: Unexpected error That's an unknown error. That is because *_max values depend on their respective non-_max values. QEMU even says that in the error message sent as a response to the monitor command: "error": {"class": "GenericError", "desc": "bps_max/iops_max require corresponding bps/iops values"} the problem was that we didn't know that and there was no check for it. Adding such check makes sure that there will be less confused users. 2016-06-09 Pavel Hrdina vnc: add support for listen type none 2016-06-09 Pavel Hrdina spice: introduce listen type none This new listen type is currently supported only by spice graphics. It's introduced to make it easier and clearer specify to not listen anywhere in order to start a guest with OpenGL support. The old way to do this was set spice graphics autoport='no' and don't specify any ports. The new way is to use . In order to be able to migrate to old libvirt the migratable XML will be generated without the listen element and with autoport='no'. Also the old configuration will be automatically converted to the this listen type. 2016-06-09 Pavel Hrdina spice: introduce spice_auto_unix_socket config option spice: add support for listen type socket Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1335832 qemu_capabilites: add QEMU_CAPS_SPICE_UNIX Add a new capability to detect support of unix sockets for spice graphics. 2016-06-09 Pavel Hrdina vnc: add support for listen type 'socket' VNC graphics already supports sockets but only via 'socket' attribute. This patch coverts that attribute into listen type 'socket'. For backward compatibility we need to handle listen type 'socket' and 'socket' attribute properly to support old XMLs and new XMLs. If both are provided they have to match, if only one of them is provided we need to be able to parse that configuration too. To not break migration back to old libvirt if the socket is provided by user we need to generate migratable XML without the listen element and use only 'socket' attribute. 2016-06-09 Pavel Hrdina graphics: introduce new listen type 'socket' 2016-06-09 Pavel Hrdina vnc: move generation of socket path to qemuProcessGraphicsSetupListen This moves the socket generation if "vnc_auto_unix_socket" is set. It also fixes a bug with this config option that we should auto-generate socket path only if listen type is address and there is no address specified. 2016-06-09 Pavel Hrdina vnc: rename socketAutogenerated to socketFromConfig Even though it's auto-generated it's based on qemu.conf option and listen type address already uses "fromConfig" to carry this information. Following commits will convert the socket to listen element so this rename is required because there will be also an option to get socket auto-generated independently on the qemu.conf option. qemu_command: move websocket code into else part for address listen There is no need to check again for vnc socket. 2016-06-09 Martin Kletzander qemu: Generate channel target paths on hotplug as well Since commit 714080791778e3dfbd484ccb3953bffd820b8ba9, qemu agent channel cannot be plugged in because we won't generate its path automatically. Let's not only fix that, but also add tests for it so next time it's checked for. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1322210 2016-06-09 Martin Kletzander qemuhotplugtest: Test live data Until now, the only hot thing in this test was the name. That's because we set the id to '-1' before every test. With this change, we test the hotplug on live domains as the name suggests and as it should be. qemu: Move channel path generation out of command creation Put it into separate function called qemuDomainPrepareChannel() and call it from the new qemuProcessPrepareDomain(). 2016-06-09 Jiri Denemark cputest: Rename nehalem-force to penryn-force The actual CPU model in the data files is Penryn which makes the file name look rather strange. Well, one of them contains Nehalem, but that's a bug which will be fixed soon. cpu: Add ARAT x86 CPU feature Implemented in QEMU by commit 28b8e4d0bf93ba176b4b7be819d537383c5a9060. cpu: Add x86 feature flags for CPUID leaf 0xd, sub leaf 1 This was implemented in QEMU by commit 0bb0b2d2fe7f645dda. cpu: Sort CPU map features on eax_in As a side effect this changes the order of CPU features in XMLs generated by libvirt, but that's not a big deal since the order there is insignificant. cpu: Shorten eax_in values in CPU map For two reasons: - 0x00000001 is very similar to 0x80000001, but 0x01 is visually different - 0x01 format is consistent with CPUID manual 2016-06-09 Jiri Denemark cpu_x86: Add full support for ecx_in CPUID parameter This patch makes our CPUID handling code up-to-date with the current specification found in Intel® 64 and IA-32 Architectures Developer's Manual: Vol. 2A http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html 2016-06-09 Jiri Denemark cpu_x86: Prepare for ecx_in CPUID parameter CPUID instruction normally takes its parameter from EAX, but sometimes ECX is used as an additional parameter. This patch prepares the x86 CPU driver code for the new 'ecx_in' CPUID parameter. qemumonitorjsontest: Add getcpu test data tests: Add CPU detection test for Intel Xeon X5460 tests: Add CPU detection test for Intel Xeon W3520 tests: Add CPU detection test for Intel Xeon E7-4820 tests: Add CPU detection test for Intel Xeon E5-2650 tests: Add CPU detection test for Intel Xeon E5-2630 tests: Add CPU detection test for Intel Xeon E3-1245 tests: Add CPU detection test for Intel Xeon 5110 tests: Add CPU detection test for AMD Phenom II X4 B95 tests: Add CPU detection test for Intel Pentium P6100 tests: Add CPU detection test for AMD Opteron 6282 SE tests: Add CPU detection test for AMD Opteron 6234 tests: Add CPU detection test for AMD Opteron 2350 tests: Add CPU detection test for AMD Opteron 1352 tests: Add CPU detection test for AMD FX 8150 tests: Add CPU detection test for Intel Core2 Quad Q9500 tests: Add CPU detection test for Intel Core2 Duo E6850 tests: Add CPU detection test for Intel Core i7-5600U tests: Add CPU detection test for Intel Core i7-4600U tests: Add CPU detection test for Intel Core i7-3770 tests: Add CPU detection test for Intel Core i7-3740QM tests: Add CPU detection test for Intel Core i7-3520M tests: Add CPU detection test for Intel Core i7-2600 tests: Add CPU detection test for Intel Core i5-6600 tests: Add CPU detection test for Intel Core i5-4670T tests: Add CPU detection test for Intel Core i5-2540M tests: Add CPU detection test for Intel Core i5-2500 tests: Add CPU detection test for Intel Atom N450 tests: Add CPU detection test for Intel Atom D510 tests: Add CPU detection test for AMD A10-5800K 2016-06-09 Jiri Denemark tests: Add CPU detection tests So far we only test CPUID -> CPU def conversion on artificial CPUID data computed from another CPU def. This patch adds the infrastructure to test this conversion on real data gathered from a host CPU and two helper scripts for adding new test data: - cpu-gather.sh runs cpuid tool and qemu-system-x86_64 to get CPUID data from the host CPU; this is what users can be asked to run if they run into an issue with host CPU detection in libvirt - cpu-parse.sh takes the data generated by cpu-gather.sh and creates data files for CPU detection tests The CPUID data queried from QEMU will eventually switch to the format used by query-host-cpu QMP command once QEMU implements it. Until then we just spawn QEMU with -cpu host and query the guest CPU in QOM. They should both provide the same CPUID results, but query-host-cpu does not require any guest CPU to be created by QEMU. 2016-06-09 Jiri Denemark cpu_x86: Refactor internal KVM features The internal features are only used in explicit checks with cpuHasFeature. Loading them into the CPU map is dangerous since the features may accidentally be reported to users when decoding CPUID data. cpu_ppc64: Avoid unnecessary pointer to virCPUppc64Data virCPUData and struct ppc64_model structures contained a pointer to virCPUppc64Data, which was not very nice since the real data were accessible by yet another level of pointers from virCPUppc64Data. cpu_x86: Avoid unnecessary pointers to virCPUx86Data virCPUData, virCPUx86Feature, and virCPUx86Model all contained a pointer to virCPUx86Data, which was not very nice since the real CPUID data were accessible by yet another pointer from virCPUx86Data. Moreover, using virCPUx86Data directly will make static definitions of internal CPU features a bit easier. tests: Create simple monitor in qemuMonitorTestNewFromFile The current version uses the first JSON reply from the file as monitor greeting. With the new parameter the caller can now request a simple test monitor to be created, which uses an artificial greeting and uses all JSON strings from the file as regular replies. tests: Fix "Reponse" typo tests: Introduce qemuMonitorTestNewFromFile It's a convenient wrapper around qemuMonitorTestNew which feeds the test monitor with QMP replies from a specified file. 2016-06-09 Jiri Denemark qemu: Refactor qemuMonitorJSONGetCPUx86Data This patch splits qemuMonitorJSONGetCPUx86Data in three functions: - qemuMonitorJSONCheckCPUx86 checks if QEMU supports reporting CPUID features for a guest CPU - qemuMonitorJSONParseCPUx86Features parses CPUID features from a JSON array - qemuMonitorJSONGetCPUx86Data gets the requested guest CPU property from QOM and uses qemuMonitorJSONParseCPUx86Features to parse it 2016-06-09 Jiri Denemark cpu_x86: Rename CPUID function to eax_in CPUID instruction normally takes its parameter from EAX, but sometimes ECX is used as an additional parameter. Let's rename 'function' to 'eax_in' in preparation for adding 'ecx_in'. cpu: Detect arch when parsing CPU data A CPU data XML file already contains the architecture, let the parser use it to detect which CPU driver should be used to parse the rest of the file. cpu_x86: Fix CPU data parser The formatter uses /cpudata/cpuid elements and the parser should really do the same. cpu_x86: Propagate vendor to guest's virCPUData When computing CPU data for a given guest CPU we should set CPUID vendor bits appropriately so that we don't lose the vendor when transforming CPU data back to XML description. 2016-06-09 Pavel Hrdina virsh-domain: fix memory leak in cmdDomDisplay 2016-06-09 Ján Tomko Fix typo in virNetDevGetEthtoolGFeatures stub s/ATTRIBUGE/ATTRIBUTE/ Reported-by: Olaf Hering 2016-06-08 Tomáš Ryšavý tests: Rename virtTestMain to virTestMain. This function doesn't follow our convention of naming functions. tests: Rename virtTestErrorFuncQuiet to virTestErrorFuncQuiet. This function doesn't follow our convention of naming functions. tests: Rename virtTestCounterNext to virTestCounterNext. This function doesn't follow our convention of naming functions. tests: Rename virtTestCaptureProgramOutput to virTestCaptureProgramOutput. This function doesn't follow our convention of naming functions. tests: Rename virtTestDifferenceBin to virTestDifferenceBin. This function doesn't follow our convention of naming functions. tests: Rename virtTestCaptureProgramExecChild to virTestCaptureProgramExecChild. This function doesn't follow our convention of naming functions. tests: Rename virtTestDifferenceFullInternal to virTestDifferenceFullInternal. This function doesn't follow our convention of naming functions. tests: Rename virtTestDifferenceFullNoRegenerate. This function doesn't follow our convention of naming functions. tests: Rename virtTestQuiesceLibvirtErrors to virTestQuiesceLibvirtErrors. This function doesn't follow our convention of naming functions. tests: Rename virtTestUseTerminalColors to virTestUseTerminalColors. This function doesn't follow our convention of naming functions. tests: Rename virtTestLogContentAndReset to virTestLogContentAndReset. This function doesn't follow our convention of naming functions. tests: Rename virtTestCounterReset to virTestCounterReset. This function doesn't follow our convention of naming functions. tests: Rename virtTest00MActive to virTest00MActive. This function doesn't follow our convention of naming functions. tests: Rename virtTestClearCommandPath to virTestClearCommandPath. This function doesn't follow our convention of naming functions. Rename virtTestDifferenceFull to virTestDifferenceFull. This function doesn't follow our convention of naming functions. tests: Rename virtTestCompareToFile to virTestCompareToFile. This function doesn't follow our convention of naming functions. Rename virtTestLoadFile to virTestLoadFile. This function doesn't follow our convention of naming functions. Rename virtTestDifference to virTestDifference. This function doesn't follow our convention of naming functions. tests: Rename virtTestRun to virTestRun. This function doesn't follow our convention of naming functions. 2016-06-08 Ján Tomko virschematest: call va_end even on OOM Jump to cleanup if virAsprintf fails. 2016-06-08 Jovanka Gulicoska node_device: Replace VIR_ERROR with standard vir*Error in state driver init 2016-06-08 Michal Privoznik virsh-network: Avoid possible NULL deref in cmdNetworkDHCPLeases Problem is, localtime_r() returns a pointer to converted time or NULL in case of an error. But checking the glibc sources, error will occur iff a NULL has been passed as an either of arguments the function takes. But GCC fails to see that: ../../tools/virsh-network.c: In function 'cmdNetworkDHCPLeases': ../../tools/virsh-network.c:1370:12: error: potential null pointer dereference [-Werror=null-dereference] ts = *localtime_r(&expirytime_tmp, &ts); ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ cc1: all warnings being treated as errors 2016-06-08 Ján Tomko lxc: simplify lxcDomainGetBlkioParameters Replace all the repetitive code by using virDomainGetBlkioParametersAssignFromDef, similar to what commit 9f50f6e did in the qemu driver. Export virDomainGetBlkioParametersAssignFromDef Move qemuDomainGetBlkioParametersAssignFromDef into domain_conf and export it, to allow reuse in the LXC driver. Use virDomainObjGetDefs in lxcDomainGetBlkioParameters Use virDomainObjGetDefs in lxcDomainSetBlkioParameters Remove yet another usage of virDomainLiveConfigHelperMethod along with an sa_assert that helped clang understand the code flow. Use virDomainObjGetDefs in lxcDomainGetMemoryParameters Instead of virDomainLiveConfigHelperMethod. 2016-06-08 Ján Tomko Use virDomainObjGetDefs in lxcDomainGetSchedulerParametersFlags On LXC domain startup we have already called virDomainObjSetDefTransient to fill vm->newDef. There is no need to call virDomainLiveConfigHelperMethod which has the ability to fill newDef if it's NULL. 2016-06-08 Ján Tomko Use virDomainObjGetDefs in lxcDomainSetSchedulerParametersFlags On LXC domain startup we have already called virDomainObjSetDefTransient to fill vm->newDef. There is no need to call virDomainLiveConfigHelperMethod which has the ability to fill newDef if it's NULL. 2016-06-08 Ján Tomko Use virDomainObjGetDefs in lxcDomainSetMemoryFlags On LXC domain startup we have already called virDomainObjSetDefTransient to fill vm->newDef. There is no need to call virDomainLiveConfigHelperMethod which has the ability to fill newDef if it's NULL. 2016-06-08 Ján Tomko lxc: rename vmdef to persistentDef A few functions using virDomainLiveConfigHelperMethod use the generic name 'vmdef' to point to the persistent definition. Use persistentDef and/or persistentDefCopy to make its purpose obvious. 2016-06-08 Daniel P. Berrange Use @SYSTEM priority for TLS on Fedora >= 21 In Fedora >= 21, there is a new crypto priority framework that sets TLS policies globally for all apps. To activate this with GNUTLS we must request "@SYSTEM" instead of the traditional "NORMAL" string. The '@' causes gnutls todo a lookup in its config file for the 'SYSTEM' keyword entry. 2016-06-08 Daniel P. Berrange remote: allow TLS priority to be customized Support reading the TLS priority from the client configuration file via the "tls_priority" config option, eg $ cat $HOME/.config/libvirt/libvirt.conf tls_priority="NORMAL:-VERS-SSL3.0" 2016-06-08 Daniel P. Berrange Pass config file object through to driver open methods The virConnectOpenInternal method opens the libvirt client config file and uses it to resolve things like URI aliases. There may be driver specific things that are useful to store in the config file too, so rather than have them re-parse the same file, pass the virConfPtr down to the drivers. 2016-06-08 Daniel P. Berrange remote: allow TLS protocol/cipher priority override in URI Add support for a "tls_priority" URI parameter in remote driver URIs. eg qemu+tls://localhost/session?tls_priority=NORMAL:-VERS-SSL3.0 2016-06-08 Daniel P. Berrange libvirtd: add config option for TLS priority Add a "tls_priority" config option to /etc/libvirt/libvirtd.conf to allow the administrator to override the built-in default setting. This only affects the server side configuration. rpc: allow priority string to be passed to TLS context Extend the virNetTLSContextNew* constructors to allow the TLS priority string to be passed in, overriding the compile time default. 2016-06-08 Daniel P. Berrange configure: allow setting default TLS priority string Currently libvirt calls gnutls_set_default_priority() which on old systems resolves to "NORMAL" while new systems it resolves to "@SYSTEM". Either way, this is a global default that is identical across all apps. We want to allow distros to flexibility to define a custom default string for libvirt priority, so add a --tls-priority=STRING flag to configure to enable this to be set. It is expected that distros would use this when creating RPM/Deb/etc packages, according to their preferred crypto handling policies. 2016-06-08 Daniel P. Berrange rpc: set gnutls log function at global init time Currently we set the gnutls log function when creating a TLS context, however, the setting is in fact global, not per context. So we should be setting it when we first call gnutls_global_init() instead. tls: remove support for gnutls 1.x.x, require 2.2.0 We need to use the gnutls_priority_set_direct method which was not introduced until 2.1.7, so bump version to 2.2.0 which is the first stable release with it included. This release dates from Dec 2007 so it is reasonable to ditch support for the 1.x.x series for gnutls releases entirely. 2016-06-08 Pavel Hrdina virsh: domdisplay: if listen is 0.0.0.0 or [::] print address from URI Currently if a guest has listen address 0.0.0.0 or [::] and you run "virsh domdisplay $domain" you always get "spice://localhost:$port". We want to print better address if someone is connected from a different computer using "virsh -c qemu+ssh://some.host/system". This patch fixes the behavior of virsh to print in this case "spice://some.host:$port". Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1332446 2016-06-08 Pavel Hrdina qemu_process: don't print empty line if qemu exits without any error Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1335617 qemu_monitor: rephrase error message if qemu closes monitor 2016-06-08 Michal Privoznik virschematest: Link with libxml2 We use libxml2 APIs in the test (e.g. xmlFreeDoc) but not link with -lxml2 which can cause problems: /usr/bin/ld: virschematest.o: undefined reference to symbol 'xmlFreeDoc@@LIBXML2_2.4.30' //usr/lib/x86_64-linux-gnu/libxml2.so.2: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status Makefile:4702: recipe for target 'virschematest' failed Reported-by: Katerina Koukiou 2016-06-08 Michal Privoznik virschematest: Access the right directory containing XMLs So the story goes like this. The testSchemaDirs() function is called with: a) the schema file, b) list of the directories that contains XMLs documents that should be checked against the schema file from a). However, the directories in the list are really just their names and it's up to testSchemaDirs to construct the absolute path and call testSchemaDir() which then does the actual validation. The absolute path is constructed, but never actually used (maybe due to a typo). Thus a VPATH build is broken. 2016-06-08 Ján Tomko Introduce virschematest Instead of calling xmllint via a shell script, use our virXMLValidator API to do it directly via libxml. Introduce virXMLValidatorValidate Split out the code for XML validation into a new function. Introduce virXMLValidatorInit Split out all the code initializing the validator to a separate function. Introduce virXMLValidatorFree Split out the code cleaning up the validator. Introduce virXMLValidator structure Store all the data related to RNG validation in one structure to allow splitting virXMLValidateAgainstSchema. 2016-06-08 Peter Krempa qemu: process: Call disk startup policy check after cloning domain def In commit 1e38ef72 the disk startup policy check was moved prior to the call to virDomainObjSetDefTransient which dropped the disk from the config rather than the def to be started which is a bug. Additionally we'd not report the disk change event for this since the disk aliases were not set at that point. Finally 'volume' based disks would not work with startup policy too. Fix it by moving it back after the definition is copied, aliases are assigned and disk sources are translated. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1341415 2016-06-08 Peter Krempa qemu: domain: Sanitize return value handling in disk presence checker One of the functions is returning always 0 and the second one uses unnecessary labels. 2016-06-08 Peter Krempa qemu: driver: Unset log file watcher after restoring a VM save file qemuProcessStart does not unset the infrastructure that retrieves errors from the qemu log file in case of migration. As this wasn't handled properly in qemuDomainSaveImageStartVM we kept the logging context/fd open for the lifetime of the VM rather than closing it after it's not needed. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1325080 2016-06-07 Philipp Hahn test: Remove executable permission from Xen xm files The configuration files are not directly executable as they don't have as hash-bang line. 2016-06-07 Philipp Hahn xen: test for driver=tap2 sub-type in xen-xm tap2 only handles 'aio', but not 'raw', which must be explicitly given: XML:raw needs to be translated to XM:aio for 'tap' and 'tap2' Xen drivers. 2016-06-07 Philipp Hahn xen: Also add sub-type for driver=tap2 in xen-xm tap2 only handles 'aio', but not 'raw', which must be explicitly given: | $ virsh domxml-to-native yyy.xml > yyy.xm | $ xm new yyy.xm | Error: tap:/srv/xen/xxx.img not a valid disk type | $ sed -i -e 's/tap2:/&aio:/' yyy.xm | $ xm new yyy.xm Fix reading and writing "xen-xm" format for "tap2" by handling it the same as "tap". 2016-06-07 Jovanka Gulicoska qemu: Replace VIR_ERROR with standard vir*Error in state driver init xen: Replace VIR_ERROR with standard vir*Error in state driver init uml: Replace VIR_ERROR with standard vir*Error in state driver init 2016-06-07 Peter Krempa qemu: migration: Add VM log entry on start of migration Note the start of migration of a qemu process to the VM log file for possible debug purposes. 2016-06-07 Peter Krempa qemu: process: Append the "shutting down" message using the new APIs Use qemuDomainLogAppendMessage rather than attempting to open a new logging context with file descriptors. The new approach allows to log the message even if qemu is still running at that point which appens during migration finish phase where qemuProcessStop is killing qemu. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1312188 2016-06-07 Peter Krempa qemu: domain: Implement helper for one-shot log entries to the VM log file Along with the virtlogd addition of the log file appending API implement a helper for logging one-shot entries to the log file including the fallback approach of using direct file access. This will be used for noting the shutdown of the qemu proces and possibly other actions such as VM migration and other critical VM lifecycle events. 2016-06-07 Peter Krempa log: daemon: Add remote protocol handling for the log appending API Implement the RPC dispatcher and caller for the new API. 2016-06-07 Peter Krempa log: handler: Add new API to append to logging files For logging one-shot entries to the VM log file it's quite a waste to hold open the file descriptor for logging that is provided by the current API. This new API will be ideal for logging one-shot entries to the file e.g. at the point when we shut the VM down rather than having to add the whole file-descriptor infrastructure. Additionally this will allow to add the messages even after restart of libvirtd since virtlogd doesn't allow to obtain a regular context with filedescriptors while the VM is still active. 2016-06-07 John Ferlan util: Perform proper virRandomBytes return value checking Document the return value of virRandomBytes as 0 or some errno value and then make sure all callers make the proper checks. 2016-06-07 Ján Tomko Reindent virNetDevSendEthtoolIoctl Reuse the socket in virNetDevGetFeatures This speeds up node_device_udev driver startup 11x. Return bool in virNetDevFeatureAvailable Simplify the logic Split out virNetDevGetEthtoolGFeatures Move out the code depending on HAVE_DECL_ETHTOOL_GFEATURES. Split out virNetDevGetEthtoolFeatures Split out the features that we probe via various ethtool commands and ETHTOOL_GFLAGS. Move struct elem out of virNetDevGetFeatures Rename struct elem to virNetDevEthtoolFeatureCmd and move it out of the function to allow reusing it. Reindent comment of virNetDevFeatureAvailable 2016-06-07 Peter Krempa qemu: process: Allow VIR_QEMU_PROCESS_START_NEW in qemuProcessLaunch The new flag was not added to virCheckFlags in commit '0d1c17aa' causing a regression where VMs were not able to start. 2016-06-07 John Ferlan util: Alter virCryptoEncryptData for non GNUTLS builds Rather than intermixing the ATTRIBUTE_UNUSED - use HAVE_GNUTLS_CIPHER_ENCRYPT for the whole function instead. 2016-06-07 Ján Tomko Add nomatch filters when enumerating udev devices Filter out some subsystems we are not interested in. node_device_udev: rename labels to cleanup Instead of the custom out and out_unlock. node_device_udev: remove unnecessary ret variables Remove ret variables and labels from functions where there is no cleanup to be done. 2016-06-07 Peter Krempa qemu: Move check that validates 'min_guarantee' to qemuDomainDefValidate Introduce a validation callback for qemu and move checking of min_guarantee to the new callback. conf: Move validation of disk LUN device to the appropriate place Now with the proper domain config validation infrastructure the check can be moved to a place that doesn't make domains vanish. 2016-06-07 Peter Krempa conf: Move disk info validator to the domain conf validator Since it will not be called from outside of conf we can unexport it too if we move it to the appropriate place. Test suite change is necessary since the error will be reported sooner now. 2016-06-07 Peter Krempa qemu: process: Call the domain config validator when starting a new VM To avoid duplicating all the checks when starting a fresh VM from a possibly unchecked config, call the domain def validator. qemu: process: Convert multiple boolean args to a single flag Validation of qemu process startup requires to know whether the process is used for a fresh VM or whether it's reloaded from a snapshot/migration. Pass this information in via a flag rather than calculating it from a bunch of bools. qemu: process: Unexport qemuProcessStartValidate conf: Add device def validation callback Similarly to the domain definition validator add a device validator. The change to the prototype of the domain validator is necessary as virDomainDeviceInfoIterateInternal requires a non-const pointer. conf: drop 'def' from struct virDomainDefPostParseDeviceIteratorData It's passed to all places along with the structure. conf: Add infrastructure for adding configuration validation Until now we weren't able to add checks that would reject configuration once accepted by the parser. This patch adds a new callback and infrastructure to add such checks. In this patch all the places where rejecting a now-invalid configuration wouldn't be a good idea are marked with a new parser flag. conf: Rename VIR_DOMAIN_DEF_PARSE_VALIDATE to VIR_DOMAIN_DEF_PARSE_VALIDATE_SCHEMA Make it obvious that the flag is controlling RNG schema validation. 2016-06-07 Ján Tomko node_device_udev: remove yoda condition udevSetupSystemDev: return if allocation fails There is no cleanup to be done. Reformat udevProcessRemoveableMedia Remove unnecessary ret variable and return early if we have no media to save on indentation. udevProcessStorage: trim all whitespace from model and vendor Use virTrimSpaces instead of a custom implementation. node_device_udev: switch to using virReportError Also use the more common "Unable to initialize mutex" string and virReportSystemError instead of virStrerror. Remove PROPERTY_* constants They are no longer used. Only return two values in udevGetUintSysfsAttr Open code the call to udev_device_get_sysattr_value in the one place where it's needed. Only return two values in udevGetIntSysfsAttr Callers only check for an error or a specific integer value. Only return two values in udevGetStringSysfsAttr The callers only care for an error, and a missing attribute is simply NULL. Remove extra allocation in udevGetDeviceSysfsAttr Most of the code paths free it right after converting it to an integer. Only return two values in udevGetUintProperty We only care about the failure, not a missing property. Only return two values in udevGetStringProperty There is no need to differentiate between PROPERTY_FOUND and PROPERTY_MISSING - we can just look if the string is non-NULL. 2016-06-07 Martin Kletzander qemu: Add support to QXL's max_outputs parameter Historically, we added heads=1 to videos, but for example for qxl, we did not reflect that on the command line. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1283207 2016-06-07 Martin Kletzander qemu: Check for qxl's max_outputs parameter Add capabilities for both qxl and qxl-vga devices. 2016-06-07 Ján Tomko Rewrite disk type checking in udevProcessStorage Error out on parsing errors and use a local const char pointer instead of chained ifs to check whether we found a match. Fix the return value in udevKludgeStorageType Since the switch to VIR_STRDUP this function returns 1 on success, but the caller treats any non-zero value as failure. 2016-06-07 Ján Tomko udevProcessFloppy; remove unnecessary allocation Use udevHasDeviceProperty instead of udevGetStringProperty. We do not need to copy the string since we do not need it. Also add braces around the if body, since the change made syntax check complain. 2016-06-07 Ján Tomko Move udevHasDeviceProperty earlier 2016-06-07 Ján Tomko Do not VIR_STRDUP the string in udevGetDeviceProperty Two out of three callers free it right after converting it to a number. Also change the comment at the beginning of the function, because the comment inside the function told me to. 2016-06-07 Ján Tomko Remove udevStrToLong_i Open code the error message. Remove udevStrToLong_ui Remove the debug message, open code the error in the two udevGetUint callers and use a more specific error in SCSI and PCI processing. 2016-06-07 Ján Tomko Remove udevStrToLong_ull The wrapper adds an error message or a debug log. Since we already log the properties we get from udev as strings, there is no much use for the debug logs. Open code the error message and delete the function. 2016-06-07 Ján Tomko Rewrite usage of StrToLong_ui in udevProcess{PCI,SCSI} Use virStrToLong_ui instead of udevStrToLong_ui, reformat the code and report a more specific error message. udevProcessSCSIHost: use STRSKIP Instead of separating it into STRPEFIX and str + strlen. 2016-06-07 Ján Tomko udevGetDMIData: remove unused variable A variable without use is pointless. Remove it, since we have no use for it. 2016-06-07 Ján Tomko Assign node device driver private data earlier Do not call nodeStateCleanup on early initialization error If we have not allocated driver yet, there is nothing to cleanup. Reformat nodeStateCleanup Remove the ret variable and return early if there is no driver. node_device_udev: initialize libpciaccess after the driver lock This will simplify cleanup. Split out pciaccess (de)initialization Move pci_system_init and pci_system_cleanup into separate functions, to make the conditional compilation easier to read. 2016-06-07 Ján Tomko Initialize ret to -1 in nodeStateInitialize Most of the code paths had to reset it to -1 and returning 0 was only possible if we made it to the end of the function. Initialize it to -1 and only set it to 0 if we reach the end, as we do in most of libvirt code. 2016-06-07 Daniel P. Berrange systemd: directly notify systemd instead of using sd_notify The sd_notify method is used to tell systemd when libvirtd has finished starting up. All it does is send a datagram containing the string parameter to systemd on a UNIX socket named in the NOTIFY_SOCKET environment variable. Rather than pulling in the systemd libraries for this, just code the notification directly in libvirt as this is a stable ABI from systemd's POV which explicitly allows independant implementations: See "Reimplementable Independently" column in the "$NOTIFY_SOCKET Daemon Notifications" row: https://www.freedesktop.org/wiki/Software/systemd/InterfacePortabilityAndStabilityChart/ Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1314881 2016-06-06 John Ferlan qemu: Move and rename qemuBuildObjectCommandlineFromJSON Move the module from qemu_command.c to a new module virqemu.c and rename the API to virQEMUBuildObjectCommandline. This API will then be shareable with qemu-img and the need to build a security object for luks support. 2016-06-06 John Ferlan storage: Create helper to set input for CreateQemuImg code Create helper virStorageBackendCreateQemuImgSetInput to set the input storage: Split out a helper for encryption checks Split out a helper from virStorageBackendCreateQemuImgCmdFromVol to check the encryption - soon a new encryption sheriff will be patroling and that'll mean all sorts of new checks. storage: Split out setting default secret for encryption Split the qcow setting of encryption secrets into a helper util: Clean up code formatting in virstorageencryption Bring style more in line with more recent code. 2016-06-06 Michal Privoznik docs: Document our event loop I was asked the other day what's event loop and how libvirt uses it. Well, I haven't found any good sources on the Internet so I thought of writing the documentation on my own. 2016-06-06 Ján Tomko Do not check for domain liveness in virDomainObjSetDefTransient Remove the live attribute and mark the definition as transient whether the domain is runing or not. There were only two callers left calling with live=false: * testDomainStartState, where the domain already is active because we assigned vm->def->id just a few lines above the call * virDomainObjGetPersistentDef, which now only calls virDomainObjSetDefTransient for an active domain 2016-06-06 Ján Tomko Check if the domain is active in virDomainObjGetPersistentDef Calling virDomainObjSetDefTransient with live=false is a no-op on an inactive domain. Only call it on an active domain, since this is the only place using the live bool. 2016-06-06 Ján Tomko Clean up redundant usage of virDomainObjSetDefTransient Commit 45ec297d from November 2010: Make state driver device hotplug/update actually transient added virDomainObjSetDefTransient calls to the domain startup function in several drivers. In November 2011, commit 8866eed: Set aliases for LXC/UML console devices added a call earlier in the startup function, without removing the existing ones. Also, in the UML driver it seems the function never did anything useful - vm->def->id is set asynchronnously in umlNotifyEvent. At the time of calling virDomainObjSetDefTransient with live=false, vm->def->id was likely still -1, making the call a no-op. 2016-06-06 Ján Tomko Post-release version bump to 1.3.6 2016-06-04 Daniel Veillard Release of libvirt-1.3.5 * docs/news.html.in: update with 1.3.5 data 2016-06-03 Daniel P. Berrange Refresh po files from zanata 2016-06-03 Martin Kletzander Fix building with -Og When building using -Og, gcc sees that some variables can be used uninitialized It can be debatable whether it is possible with our codeflow, but functions should be self-contained and initializations are always good. The return instead of goto is due to actualType being used in the cleanup. 2016-06-03 Michal Privoznik virPerfEventIsEnabled: Don't crash on shut off domains So imagine the following. You connect read only to a daemon and try to fetch stats for a shut off domain, e.g.: virsh -r domstats $dom but all of a sudden, virsh instead of printing the stats throws the following error at you: error: Disconnected from qemu:///system due to I/O error error: End of file while reading data: Input/output error The daemon crashed. This is its backtrace: #0 0x00007fa43e3751a8 in virPerfEventIsEnabled (perf=0x0, type=VIR_PERF_EVENT_MBMT) at util/virperf.c:241 #1 0x00007fa424a9f042 in qemuDomainGetStatsPerf (driver=0x7fa3f4022a30, dom=0x7fa3f40e24c0, record=0x7fa41c000e20, maxparams=0x7fa4360b38d0, privflags=1) at qemu/qemu_driver.c:19110 #2 0x00007fa424a9f2e7 in qemuDomainGetStats (conn=0x7fa41c001b20, dom=0x7fa3f40e24c0, stats=127, record=0x7fa4360b3970, flags=1) at qemu/qemu_driver.c:19213 #3 0x00007fa424a9f672 in qemuConnectGetAllDomainStats (conn=0x7fa41c001b20, doms=0x7fa41c0017f0, ndoms=1, stats=127, retStats=0x7fa4360b3a50, flags=0) at qemu/qemu_driver.c:19303 #4 0x00007fa43e4e15f6 in virDomainListGetStats (doms=0x7fa41c0017f0, stats=0, retStats=0x7fa4360b3a50, flags=0) at libvirt-domain.c:11615 Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f28d1a38700 (LWP 16154)] 0x00007f28da4fa1a8 in virPerfEventIsEnabled (perf=0x0, type=VIR_PERF_EVENT_MBMT) at util/virperf.c:241 241 return event->enabled; Problem is, shut off domains don't have priv->perf allocated. Therefore if in frame #1 qemuDomainGetStatsPerf() tries to check if perf events are enabled, NULL is passed to virPerfEventIsEnabled() which due to some incredible implementation dereference it. Fix this by checking whether passed object is not NULL. 2016-06-03 Michal Privoznik Drop virPerfGetEventFd This function is not used anywhere. Moreover, the code that would use lives in virperf.c and therefore has access to the FD anyway. Well, for instance virPerfReadEvent is doing just that. 2016-06-03 Michal Privoznik virDomainChrGetDomainPtrsInternal: Return an integer There's this problem on the recent gcc-6.1: In file included from conf/domain_conf.c:37:0: conf/domain_conf.c: In function 'virDomainChrPreAlloc': conf/domain_conf.c:14109:35: error: potential null pointer dereference [-Werror=null-dereference] return VIR_REALLOC_N(*arrPtr, *cntPtr + 1); ^~ ./util/viralloc.h:158:73: note: in definition of macro 'VIR_REALLOC_N' # define VIR_REALLOC_N(ptr, count) virReallocN(&(ptr), sizeof(*(ptr)), (count), \ ^~~~~ conf/domain_conf.c: In function 'virDomainChrRemove': conf/domain_conf.c:14133:21: error: potential null pointer dereference [-Werror=null-dereference] for (i = 0; i < *cntPtr; i++) { ^~~~~~~ GCC basically fails to see, that the virDomainChrGetDomainPtrsInternal will never actually return NULL because it's never called over a domain char device with _LAST type. But to make it shut up, lets turn this function into returning an integer and check in the callers if a zero value value was returned. 2016-06-02 Michal Privoznik virDomainFormatSchedDef: Avoid false positive NULL dereference Okay, I admit that our code here is complex. It's not easy to spot that NULL deref can't really happen here. So it's no wonder that a dumb compiler fails to see all the connections and produces the following errors: CC conf/libvirt_conf_la-domain_conf.lo conf/domain_conf.c: In function 'virDomainDefFormatInternal': conf/domain_conf.c:22162:22: error: potential null pointer dereference [-Werror=null-dereference] if (sched->policy == i) ~~~~~^~~~~~~~ cc1: all warnings being treated as errors 2016-06-02 Michal Privoznik ppc64Compute: Avoid possible NULL dereference cpu/cpu_ppc64.c: In function 'ppc64Compute': cpu/cpu_ppc64.c:620:27: error: potential null pointer dereference [-Werror=null-dereference] if (STRNEQ(guest_model->name, host_model->name)) { ~~~~~~~~~~~^~~ cpu/cpu_ppc64.c:620:9: note: in expansion of macro 'STRNEQ' if (STRNEQ(guest_model->name, host_model->name)) { ^~~~~~ cc1: all warnings being treated as errors virNetDevBridgeGet: Don't require users to virNetDevSetupControl So far, this function has just three callers. Two of them call virNetDevSetupControl to create a socket that we can then optionally use for ioctl() to fetch data. However, querying sysfs is preferred. Therefore it doesn't make much sense to require users to set up the socket if they don't even know it will be used in favour of sysfs. We can set up the socket iff we need to. 2016-06-01 Laine Stump network: restart dnsmasq after adding/removing txt and srv records Although dns host records are stored in a separate configuration file that is reread by dnsmasq when it receives a SIGHUP, the txt and srv records are directly in the dnsmasq .conf file which can't be reread after initial dnsmasq startup. This means that if an srv or txt record is modified in a network config, libvirt needs to restart the dnsmasq process rather than just sending a SIGHUP. This was pointed out in a question in https://bugzilla.redhat.com/show_bug.cgi?id=988718 , but no separate BZ was filed. 2016-06-01 Pavel Hrdina QXL: fix reloading of vram64 attribute Commit b4a5fd95 introduced vram64 attribute for QXL video device but there were two issues. Only function qemuMonitorJSONUpdateVideoVram64Size should update the vram64 attribute and also the value is in MiB, not in B. 2016-05-31 Michal Privoznik esxStorageVolGetXMLDesc: Lookup SCSI lun properly So the idea is as follows: firstly we obtain a list of all the luns, then iterate over it trying to find the one we want to work with and after all the iterations we detect whether we have found something. Now, the last check is broken, because it compares a value form previous iteration, not the one we've just been through. Then, when computing md5 sum of lun's UUID, we use wrong variable again. Well, @hostScsiDisk which is type of esxVI_HostScsiDisk extends esxVI_ScsiLun type so they both have the uuid member, but it just doesn't feel right to access the data via two different variables in one function call. 2016-05-31 Michal Privoznik qemuMonitorTextGetAllBlockStatsInfo: Fix line validation There's a bug in the function. We expect the following format for the data we are parsing here: key: value So we use strchr() to find ':' and then see if it is followed by space. But the check that does just that is slightly incorrect. 2016-05-30 Michal Privoznik virSocketAddrIsPrivate: Work on 32bits platforms Yet another one of those where signed int (or long int) is not enough. And useless to as we're aiming at unsigned anyway. ../../src/util/virsocketaddr.c: In function 'virSocketAddrIsPrivate': ../../src/util/virsocketaddr.c:289:45: error: result of '192l << 24' requires 33 bits to represent, but 'long int' only has 32 bits [-Werror=shift-overflow=] return ((val & 0xFFFF0000) == ((192L << 24) + (168 << 16)) || ^~ ../../src/util/virsocketaddr.c:290:45: error: result of '172l << 24' requires 33 bits to represent, but 'long int' only has 32 bits [-Werror=shift-overflow=] (val & 0xFFF00000) == ((172L << 24) + (16 << 16)) || ^~ cc1: all warnings being treated as errors 2016-05-30 Michal Privoznik apibuild: Substitute only pure number tokens In 38df47c9af1 I've tried to prepare our apibuild.py script for change made in 0628f3498ce (1U << 31). What I've done in the former commit was to replace \d+U in parsed tokens with \d. Problem was, my regular expression there was not quite right as it also translated VIR_123U_VAL into VIR_123_VAL. 2016-05-28 Michal Privoznik build: use gnulib's unsetenv Now that gnulib has lifted it's licensing of unsetenv, we should use it. Just like we use its counterpart - setenv, already. 2016-05-28 Michal Privoznik Turn 1<<31 into 1U<<31 Apparently, 1 << 31 is signed which in turn does not fit into a signed integer variable: ../../include/libvirt/libvirt-domain.h:1881:57: error: result of '1 << 31' requires 33 bits to represent, but 'int' only has 32 bits [-Werror=shift-overflow=] VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS = 1 << 31, /* enforce requested stats */ ^~ cc1: all warnings being treated as errors The solution is to make it an unsigned value. I've found only two such occurrences in our code base. 2016-05-28 Michal Privoznik docs: Teach apibuild to deal with (1U << 31) too The apibuild script is a terrifying beast that parses some source files of ours and produces an XML representation of them. When it comes to parsing enums we have in some header files, it tries to be clever and detect a value that an enum member has (or if it is an alias for a different member). Whilst doing that it has to deal with values we give to the members in many formats. At some places we just pass the value in decimal: VIR_DOMAIN_BLOCK_JOB_TYPE_PULL = 1, in other places, we use the aliasing: VIR_CONNECT_GET_ALL_DOMAINS_STATS_ACTIVE = VIR_CONNECT_LIST_DOMAINS_ACTIVE, and in other places bitwise shifts are used: VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS = 1 << 31, /* enforce requested stats */ The script tries to parse all of these resulting in the following tokens: "1", "VIR_CONNECT_LIST_DOMAINS_ACTIVE", "1<<31"; Then, the script tries to turn these into integers using python's eval() function. This function succeeds on the first and the last tokens. But, if we were to modify the last example so that it's of the following form: VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS = 1U << 31, /* enforce requested stats */ the token representing enum's member value will then be "1U<<31". So our parsing is good. Unfortunately, python is not aware of the difference between signed and unsigned C types, therefore eval() fails over this token and the parser falls back thinking it's an alias to another enum member. Well it's not. The solution is to transform [0-9]U into [0-9] as for our purposes here it's the same thing. 2016-05-27 Eric Blake maint: update to latest gnulib Fix a regression in checking for realpath (which caused link failures regarding duplicate rpl_canonicalize_file_name), and fix the mingw build regarding unsetenv. * .gnulib: Update to latest. 2016-05-27 Cole Robinson spec: Advertise nvram paths of official fedora edk2 builds Fedora now ships edk2 firmware in its official repos, so adapt the nvram path list to match. Eventually we can remove the nightly links as well once some integration kinks have been worked out, and documentation updated. Move the macro building into the %build target, which lets us build up a shell variable and make things a bit more readable https://bugzilla.redhat.com/show_bug.cgi?id=1335395 2016-05-27 Katerina Koukiou lxc: Fix virLXCDomainObjBeginJob position in lxcDomainSetMemoryParameters Adjust the code to perform the virLXCDomainObjBeginJob first and then the call virDomainLiveConfigHelperMethod. As Ján Tomko pointed out, in virDomainLiveConfigHelperMethod, there is a check to see if the domain is active when AFFECT_LIVE is set. Since virLXCDomainObjBeginJob unlocks the virDomainObjPtr lock, the domain could possibly be destroyed while we wait for the job and the check results would no longer be valid. 2016-05-26 Eric Blake maint: update to latest gnulib Pulls in several portability fixes, including the fact that gnulib now only works on platforms with two's complement signed integers. Also makes for a smaller delta on the next update (we are waiting on a license change to unsetenv for the sake of mingw). * .gnulib: Update to latest. * bootstrap: Resync from upstream. * tests/virstringtest.c: Drop use of obsolete probes of integer properties. 2016-05-26 Dawid Zamirski esx: do not store escaped password in esxVI_Context. This patch fixes an issue where screenshot API call was failing when the esx/vcenter password contains special characters such as apostrophee. The reason for failures was that passwords were escaped for XML and stored in esxVI_Context which was then passed to raw CURL API calls where the password must be passed in original form to authenticate successfully. So this patch addresses this by storing original passwords in the esxVI_Context struct and escape only for esxVI_Login call. 2016-05-26 Andrea Bolognani qemu: Fix error message when PCI bridge has index <= bus Commit ff2126225df0 changed the error message to be more detailed about the failure at hand; however, while the new error message claims that "bus must be <= index", the error message is displayed if "idx <= addr->bus", ie. when bus is larger than or *equal to* index. Change the error message to report the correct constraint, and format it in a way that mirrors the check exactly to make it clearer to people reading the code. The new error message reads "index must be larger than bus". Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1339900 2016-05-26 Nikolay Shirokovskiy daemon: cleanup state drivers in order reverse to init order This patch aims to fix observed crash on daemon shutdown. Main thread is in the process of state drivers cleanup, network driver is cleaned up and qemu driver is not yet. Meanwhile eof event from qemu process triggers qemuProcessStop -> networkReleaseActualDevice and crash happens as network driver is already cleaned up. 2016-05-26 Dawid Zamirski esx: use newer virtualHW version for 5.1+ hosts This is because there's a known issue where ESX will refuse to attach drives bigger than 4TB when virtualHW < 9. Therefore, to avoid that use the higher virtualHW for hosts that support it. https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2054952 2016-05-26 Dawid Zamirski esx: Add VMCI device for virtualHW >= 7 This patch fixes an issue where vMotion fails when VMCI device is not present in the vmx file. esx: add pciBridge devices when SCSI is used When a SCSI controller is present, ESX adds several pciBridge devices to vmx file. This fixes an error message where it refuses to create VM due to not enough PCI devices available. This applies only to virtualHW version >= 7. 2016-05-25 Laine Stump conf: permit auto-assignment of controller indexes Hand-entering indexes for 20 PCI controllers is not as tedious as manually determining and entering their PCI addresses, but it's still annoying, and the algorithm for determining the proper index is incredibly simple (in all cases except one) - just pick the lowest unused index. The one exception is USB2 controllers because multiple controllers in the same group have the same index. For these we look to see if 1) the most recently added USB controller is also a USB2 controller, and 2) the group *that* controller belongs to doesn't yet have a controller of the exact model we're just now adding - if both are true, the new controller gets the same index, but in all other cases we just assign the lowest unused index. With this patch in place and combined with the automatic PCI address assignment, we can define a PCIe switch with several ports like this: ... These will each get a unique index, and PCI addresses that connect them together appropriately with no pesky numbers required. 2016-05-25 Laine Stump conf: make virDomainControllerFindUnusedIndex() more generally usable Make virDomainControllerFindUnusedIndex() a global function so that it can be used outside domain_conf.c (as well as higher up in domain_conf.c itself)/ Also make its DomainDef arg a const* so that functions which only have a const* to the domain can use it. conf/qemu: make IS_USB2_CONTROLLER globally available IS_USB2_CONTROLLER() is useful in more places aside from just when assigning PCI addresses in QEMU, and is checking for enum values that are all defined in conf/domain_conf.h anyway, so define it there instead. 2016-05-25 Chunyan Liu libxl: add .domainInterfaceAddresses Add .domainInterfaceAddresses so that user can have a way to get domain interface address by 'virsh domifaddr'. Currently it only supports '--source lease'. Signed-off: Chunyan Liu 2016-05-25 Ján Tomko security: label the slic_table Add support for the slic_table to the security drivers. 2016-05-25 Ján Tomko qemu: format SLIC ACPI table command line /path/to/acpi/table/file
 will result in: -acpitable sig=SLIC,file=/path/to/acpi/table/file This option was introduced by QEMU commit 8a92ea2 in 2009. https://bugzilla.redhat.com/show_bug.cgi?id=1327537 2016-05-25 Ján Tomko conf: add to Add a new element to XML:
/path/to/acpi/table/file
 To supply a path to a SLIC (Software Licensing) ACPI table blob. https://bugzilla.redhat.com/show_bug.cgi?id=1327537 2016-05-25 Pavel Hrdina qemucapstest: replace caps-1.6.50 with updated caps-1.7.0 The qemu-1.6.50 is a beta before the new minor version, let's replace it with the release qemu-1.7.0. 2016-05-25 Peter Krempa qemu: Remove virDomainLiveConfigHelperMethod from qemuDomainSetSchedulerParametersFlags This refactor also makes a distinction between the pointer to the original definition and copied one to prevent mixups. qemu: Remove virDomainLiveConfigHelperMethod from qemuDomainSetBlockIoTune qemu: Refactor qemuDomainGetSchedulerParametersFlags Use virDomainCputune struct to store the data rather than exploding the fields and use macros to fill the typed params. conf: Change virDomainCputune member 'shares' to unsigned long long cgroup functions set and get the longer type so use it everywhere qemu: Remove virDomainLiveConfigHelperMethod from qemuDomainGetSchedulerParametersFlags qemu: Refactor qemuDomainGetBlkioParameters Get rid of lots of duplicated code. qemu: Remove virDomainLiveConfigHelperMethod from qemuDomainGetBlkioParameters qemu: Remove virDomainLiveConfigHelperMethod from qemuDomainSetMemoryParameters qemu: Refactor typed params assignment in qemuDomainGetBlockIoTune Introduce a macro to assign the parameters to avoid the for loop and shuffle around various checks for a simpler and saner function. qemu: Replace virDomainLiveConfigHelperMethod in qemuDomainGetBlockIoTune Use virDomainObjGetDefs since the API guarantees that both live and config are never set together. qemu: monitor: Remove 'supportMaxOptions' argument from qemuMonitorGetBlockIoThrottle The caller is already aware that the params are missing and the extractor is ignoring the missing ones so the parameter isn't necessary. 2016-05-25 Dawid Zamirski esx: use lsilogic adapter type in vol create. ESX will refuse to attach VMDKS that have buslogic adatper type to 64bit VMs whereas lsilogic works fine both 32bit and 64bit VMs. 2016-05-25 Jim Fehlig libxl: default to qemu driver for network disks Xen only supports network-based disks with the qemu (aka qdisk) driver. Set the driverName to 'qemu' in libxlDomainDeviceDefPostParse() if not already set. When starting a domain with network-based disks, ensure the driverName is 'qemu'. Resolves: https://bugzilla.opensuse.org/show_bug.cgi?id=981094 2016-05-25 Peter Krempa qemu: driver: Allow disk update of startupPolicy/snapshot for all disks The libvirt internal bits can be changed for disks that don't otherwise support changing media. Remove the switch statement and allow changes of non-source data for all disks. qemu: driver: Move around code to avoid need to rollback qemuDomainChangeDiskLive rolled back few changes to the disk definition if changing of the media failed. This can be avoided by moving some code around. 2016-05-25 Shivaprasad G Bhat Call qemuDomainObjEndJob when qemuCaps is null during hotplug Unref the cfg in qemuDomainAttachHostPCIDevice() 2016-05-25 John Ferlan lxc: Fix lxcDomainDestroyFlags endjob processing Commit id '15ccb0dbf' added job functions for the lxc driver; however, for shutdown and nonpersistent path, the vm was removed from the domain object list and the vm pointer cleared before the endjob. Adjust the code to perform the endjob first and then perform the ObjListRemove as long as the vm wasn't NULL. This follows more closely models from qemu and libxl Found by Coverity (FORWARD_NULL) 2016-05-25 John Ferlan qemu: Remove unused persistentAddrs Based on some digital archaeology performed by jtomko, it's been determined that the persistentAddrs variable is no longer necessary... The variable was added by: commit 141dea6bc7222107c2357acb68066baea5b26df3 CommitDate: 2010-02-12 17:25:52 +0000 Add persistence of PCI addresses to QEMU Where it was set to 0 on domain startup if qemu did not support the QEMUD_CMD_FLAG_DEVICE capability, to clear the addresses at shutdown, because QEMU might make up different ones next time. As of commit f5dd58a6088cfc6e8bd354b693d399807a8ec395 CommitDate: 2012-07-11 11:19:05 +0200 qemu: Extended qemuDomainAssignAddresses to be callable from everywhere. this was broken, when the persistentAddrs = 0 assignment was moved inside qemuDomainAssignPCIAddresses and while it pretends to check for !QEMU_CAPS_DEVICE, its parent qemuDomainAssignAddresses is only called if QEMU_CAPS_DEVICE is present. 2016-05-25 John Ferlan qemu: Remove dead code Since commit id '20a0fa8e' removed the QEMU_CAPS_DEVICE, Coverity notes that it's no longer possible to have 'addrs' be NULL when checking for a live domain since qemuDomainPCIAddressSetCreate would have jumped to cleanup if addrs was NULL. 2016-05-25 Andrea Bolognani conf: nodedev: Set PCI_PHYSICAL_FUNCTION flag more carefully Instead of setting the flag before parsing the PCI address, set it afterwards. This ensure we can never end up in a situation where the flag has been set but pci_dev.physical_function has not been filled in. 2016-05-25 Andrea Bolognani pci: Fix virPCIGetPhysicalFunction()'s callers Commit c8b1a83605e4 changed the function, making it impossible for callers to be able to tell whether a non-negative return value means "physical function address found and parsed correctly" or "couldn't find corresponding physical function". The important difference between the two being that, in the latter case, the returned pointer is NULL and should never, ever be dereferenced. In order to cope with these changes, the callers have to be updated. 2016-05-25 Andrea Bolognani pci: Document virPCIGetPhysicalFunction() pci: Initialize return location in virPCIGetPhysicalFunction() Just an extra precaution in case the function returns early due to an OOM error. 2016-05-25 Peter Krempa qemu: hotplug: wait for the tray to eject only for drives with a tray Use the detected tray presence flag to trigger the tray waiting code only if the given storage device in qemu reports to have a tray. This is necessary as the floppy device lost it's tray as of qemu commit: commit abb3e55b5b718d6392441f56ba0729a62105ac56 Author: Max Reitz Date: Fri Jan 29 20:49:12 2016 +0100 Revert "hw/block/fdc: Implement tray status" 2016-05-25 Peter Krempa qemu: hotplug: Fix error reported when cdrom tray is locked Commit 1fad65d49aae364576bd91352a001249510f8d4e used a really big hammer and overwrote the error message that might be reported by qemu if the tray is locked. Fix it by reporting the error only if no error is currently set. Error after commit mentioned above: error: internal error: timed out waiting for disk tray status update New error: error: internal error: unable to execute QEMU command 'eject': Tray of device 'drive-ide0-0-0' is not open 2016-05-25 Peter Krempa qemu: hotplug: Extract code for waiting for tray eject The code grew rather convoluted. Extract it to a separate function. qemu: process: Fix and improve disk data extraction Extract information for all disks and update tray state and source only for removable drives. Additionally store whether a drive is removable and whether it has a tray. qemu: Move and rename qemuDomainCheckEjectableMedia to qemuProcessRefreshDisks Move it to a more sane place since it's refreshing data about disks. 2016-05-25 Peter Krempa qemu: Extract more information about qemu drives Extract whether a given drive has a tray and whether there is no image inserted. Negative logic for the image insertion is chosen so that the flag is set only if we are certain of the fact. 2016-05-25 Peter Krempa qemu: Move struct qemuDomainDiskInfo to qemu_domain.h 2016-05-25 Joao Martins xenconfig: xm: check for driver on disk format When reviewing libxl vif typename series[0] I found a bug on xen-xm formatter where "virsh domxml-to-native xen-xm file.xml" can lead to a NULL dereference if the disk driver isn't specified. Fix this by checking for driver before writing/testing it down. [0] https://www.redhat.com/archives/libvir-list/2016-April/msg01434.html 2016-05-24 Laine Stump lxc: support This is identical to type='bridge', but without the "connect to a bridge" part, so it can be handled by using the same functions (and often even the same cases in switch statements), after renaming virLXCProcessSetupInterfaceBridged() to virLXCProcessInterfaceTap() and enhancing it to skip bridge-related items when brname == NULL. To be truly useful, we need to support setting the ip address on the host side veth as well as guest side veth (already supported for type='bridge'), as well as setting the peer address for both. The Device pci_0000_00_19_0 dettached Domain test started Device attached successfully error: Failed to start domain test2 error: Requested operation is not valid: PCI device 0000:00:19.0 is in use by domain test [ -- 1th time --] Device pci_0000_00_19_0 re-attached [ -- 2th time --] Device pci_0000_00_19_0 re-attached [ -- 3th time --] Device pci_0000_00_19_0 re-attached [ -- 4th time --] Device pci_0000_00_19_0 re-attached [ -- 5th time --] Device pci_0000_00_19_0 re-attached clean up Domain test destroyed Device pci_0000_00_19_0 re-attached The patch also fixes another problem, there won't be error like "qemuDomainReAttachHostdevDevices: Not reattaching active device 0000:00:19.0" in daemon log if some device is in active. As pciResetDevice and pciReattachDevice won't be called for the device anymore. This is sensible as we already reported error when preparing the device if it's active. Blindly trying to pciResetDevice & pciReattachDevice on the device and getting an error is just redundant. 2011-12-15 Osier Yang qemu: Honor the original properties of PCI device when detaching This patch fixes two problems: 1) The device will be reattached to host even if it's not managed, as there is a "pciDeviceSetManaged". 2) The device won't be reattached to host with original driver properly. As it doesn't honor the device original properties which are maintained by driver->activePciHostdevs. 2011-12-14 Wen Congyang spec: fix inverted logic on sanlock Commit d336dbdb tried to refactor sanlock to avoid building it on RHEL for architectures where it is not available, but used the wrong conditional. * libvirt.spec.in (with_sanlock): Use %ifarch, not %ifnarch. 2011-12-14 KAMEZAWA Hiroyuki virsh: support multifunction in attach-disk PCI can be specified by attach-disk but multifunction cannot be specified. Add --multifunction support. 2011-12-13 Eric Blake docs: tweak 'virsh edit' wording I was wondering why 'virsh edit' didn't support the same '--inactive' option as 'virsh dumpxml'; reading the source code showed that --inactive was already implied, and that the only way to alter a running guest rather than affecting next boot is by hot-plugging individual devices, or by something complex like saving the guest and modifying the save image. * tools/virsh.pod (define, edit): Mention behavior when guest is already running. 2011-12-13 Peter Krempa python: Fix export of virDomainSnapshotListChildrenNames Commit f2013c9dd1ce468b8620ee35c232a93ef7026fb0 added implementation of virDomainSnapshotListChildrenNames override export, but registration of the newly exported function was not added. *python/libvirt-override.c: - register export of function 2011-12-13 Lei Li Provide a helper method virDomainLiveConfigHelperMethod This chunk of code below repeated in several functions, factor it into a helper method virDomainLiveConfigHelperMethod to eliminate duplicated code based on Eric and Adam's suggestion. I have tested it for all the relevant APIs changed. 2011-12-13 Peter Krempa virsh: Print error message if argument parsing fails for cmdNodesuspend If parsing of arguments failed, virsh did silently exit returning and error state, but not specifying the possible problem. * tools/virsh: cmdNodesuspend: - error handling added 2011-12-13 Alex Jia tests: plug memory leak on linuxTestNodeInfo Detected by valgrind. Leak introduced in commit 82ff25e. * tests/nodeinfotest.c: avoid memory leak on nodeinfo test case. * how to reproduce? % cd tests && valgrind -v --leak-check=full ./nodeinfotest * actual valgrind result: ==22147== 65 bytes in 1 blocks are definitely lost in loss record 14 of 29 ==22147== at 0x4A0610F: realloc (vg_replace_malloc.c:525) ==22147== by 0x330D6FED94: __vasprintf_chk (in /lib64/libc-2.12.so) ==22147== by 0x426697: virVasprintf (stdio2.h:199) ==22147== by 0x426757: virAsprintf (util.c:1695) ==22147== by 0x41585F: linuxTestNodeInfo (nodeinfotest.c:108) ==22147== by 0x416B21: virtTestRun (testutils.c:141) ==22147== by 0x4157EA: mymain (nodeinfotest.c:140) ==22147== by 0x416217: virtTestMain (testutils.c:696) ==22147== by 0x330D61ECDC: (below main) (in /lib64/libc-2.12.so) ==22147== ==22147== LEAK SUMMARY: ==22147== definitely lost: 65 bytes in 1 blocks ==22147== indirectly lost: 0 bytes in 0 blocks ==22147== possibly lost: 0 bytes in 0 blocks ==22147== still reachable: 126,126 bytes in 1,341 blocks 2011-12-13 Osier Yang storage: Fix a potential crash when creating vol object If the vol object is newly created, it increases the volumes count, but doesn't decrease the volumes count when do cleanup. It can cause libvirtd to crash when one trying to free the volume objects like: for (i = 0; i < pool->volumes.count; i++) virStorageVolDefFree(pool->volumes.objs[i]); It's more reliable if we add the newly created vol object in the end. 2011-12-12 Eric Blake docs: document
elements in one place Improve the documentation of what forms a valid
element, since these elements appear in numerous devices. * docs/formatdomain.html.in (elementsAddress): New section. (elementsControllers, elementsUSB, elementsNICS, elementsInput) (elementsHub, elementsCharChannel, elementsSound): Refer to it. 2011-12-12 Eric Blake build: follow directory install conventions Commit 4d9e51f6 fixed a 'make uninstall' failure, but failed to follow other conventions already present in src/Makefile.am. In particular, we prefer MKDIR_P over mkdir -p, and should have a matching rmdir during uninstall for every directory created during install (the idea being that uninstall in a DESTDIR should be clean, while installation in the final system should not fail with non-empty directories left behind). * tools/Makefile.am (install-sysconfig, install-initscript) (install-systemd): Use MKDIR_P. (uninstall-sysconfig, uninstall-initscript, uninstall-systemd): Also remove directories. * daemon/Makefile.am (install-data-local, install-data-polkit) (install-logrotate, install-sysconfig, install-sysctl) (install-init-redhat, install-init-upstart, install-init-systemd) (install-data-sasl): Use MKDIR_P. (uninstall-data-polkit, uninstall-sysconfig, uninstall-sysctl) (uninstall-init-redhat, uninstall-init-upstart) (uninstall-init-systemd): Also remove directory. (uninstall-logrotate): New rule. (uninstall-local): Add uninstall-logrotate. 2011-12-12 Jiri Denemark qemu: Disable EOF processing during qemuDomainDestroy When destroying a domain qemuDomainDestroy kills its qemu process and starts a new job, which means it unlocks the domain object and locks it again after some time. Although the object is usually unlocked for a pretty short time, chances are another thread processing an EOF event on qemu monitor is able to lock the object first and does all the cleanup by itself. This leads to wrong shutoff reason and lifecycle event detail and virDomainDestroy API incorrectly reporting failure to destroy an inactive domain. Reported by Charlie Smurthwaite. 2011-12-12 Michal Privoznik virsh: Free returned MIME type string In terms of documentation to virDomainScreenshot, caller MUST free returned value. But virsh was not. 2011-12-12 Osier Yang Maint: Update AUTHORs Add Rommer in. 2011-12-12 Rommer storage: Activate/deactivate logical volumes only on local node Current "-ay | -an" has problems on pool starting/refreshing if the volumes are clustered. Rommer has posted a patch to list 2 months ago. https://www.redhat.com/archives/libvir-list/2011-October/msg01116.html But IMO we shouldn't skip the inactived vols. So this is a squashed patch by Rommer. 2011-12-12 Josh Durgin security: don't try to label network disks Network disks don't have paths to be resolved or files to be checked for ownership. ee3efc41e6233e625aa03003bf3127319ccd546f checked this for some image label functions, but was partially reverted in a refactor. This finishes adding the check to each security driver's set and restore label methods for images. 2011-12-12 Dave Allan Fix make uninstall Make uninstall currently fails with the following message: rmdir /etc/sasl2/ rmdir: failed to remove `/etc/sasl2/': Directory not empty That's fine (correct in fact) so force the command to return success with || : 2011-12-10 Laine Stump test: replace deprecated "fedora-13" machine with "pc-0.13" One of the xml tests in the test suite was created using a now-deprecated qemu machine type ("fedora-13", which was only ever valid for Fedora builds of qemu). Although strictly speaking it's not necessary to replace it with an actual supported qemu machine type (since the xml in question is never actually sent to qemu), this patch changes it to the actually-supported "pc-0.13" just for general tidiness. (Also, on some Fedora builds which contain a special patch to rid the world of "fedora-13", having it mentioned in the test suite will cause make check to fail.) 2011-12-10 Laine Stump network: don't add iptables rules for externally managed networks This patch addresses https://bugzilla.redhat.com/show_bug.cgi?id=760442 When a network has any forward type other than route, nat or none, the network configuration should be done completely external to libvirt - libvirt only uses these types to allow configuring guests in a manner that isn't tied to a specific host (all the host-specific information, in particular interface names, port profile data, and bandwidth configuration is in the network definition, and the guest configuration only references it). Due to a bug in the bridge network driver, libvirt was adding iptables rules for networks with forward type='bridge' etc. any time libvirtd was restarted while one of these networks was active. This patch eliminates that error by only "reloading" iptables rules if forward type is route, nat, or none. 2011-12-09 Michael Ellerman qemu: Prepare to cater for more general address assignment Currently qemuDomainAssignPCIAddresses() is called to assign addresses to PCI devices. We need to do something similar for devices with spapr-vio addresses. So create one place where address assignment will be done, that is qemuDomainAssignAddresses(). 2011-12-09 Michael Ellerman qemu: Add address in qemuBuildChrDeviceStr() on pseries For the PPC64 pseries machine type we need to add address information for the spapr-vty device. qemu: Use spapr-vscsi on pseries machine type On the PPC64 pseries machine type we need to use the spapr-vscsi device rather than an lsi. 2011-12-09 Eric Blake network: allow '-' in model name In QEMU PPC64 we have a network device called "spapr-vlan". We can specify this using the existing syntax for network devices, however libvirt currently rejects "spapr-vlan" in virDomainNetDefParseXML() because of the "-". Fix the code to accept "-". * src/conf/domain_conf.c (virDomainNetDefParseXML): Allow '-' in model name, and be more efficient. * docs/schemas/domaincommon.rng: Limit valid model names to match code. Based on a patch by Michael Ellerman. 2011-12-09 Michal Privoznik threadpool: Use while loop on virCondWait instead of simple 'if' statement as virCondWait can return even if associated condition was not signaled. threads: Document spurious wakeups on virCondWait 2011-12-09 Alex Jia virsh: plug memory leak on cmdDomblklist Detected by valgrind. Leak introduced in commit 88a993b: * tools/virsh.c: fix memory leak on cmdDomblklist. * how to reproduce? % valgrind -v --leak-check=full virsh domblklist * actual valgrind result: ==6573== 1,836 bytes in 1 blocks are definitely lost in loss record 110 of 124 ==6573== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==6573== by 0x330D71497D: xdr_string (in /lib64/libc-2.12.so) ==6573== by 0x4D26CED: xdr_remote_nonnull_string (remote_protocol.c:30) ==6573== by 0x4D28138: xdr_remote_domain_get_xml_desc_ret (remote_protocol.c:1418) ==6573== by 0x4D3C0C2: virNetMessageDecodePayload (virnetmessage.c:382) ==6573== by 0x4D3279F: virNetClientProgramCall (virnetclientprogram.c:382) ==6573== by 0x4D0D50B: callWithFD (remote_driver.c:4339) ==6573== by 0x4D0D5AB: call (remote_driver.c:4360) ==6573== by 0x4D16EAF: remoteDomainGetXMLDesc (remote_client_bodies.h:861) ==6573== by 0x4CF9F4F: virDomainGetXMLDesc (libvirt.c:4098) ==6573== by 0x4154D9: cmdDomblklist (virsh.c:1722) ==6573== by 0x4149E2: vshCommandRun (virsh.c:16365) ==6573== ==6573== 46,009 (352 direct, 45,657 indirect) bytes in 1 blocks are definitely lost in loss record 123 of 124 ==6573== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==6573== by 0x3318286DC6: xmlXPathNewContext (in /usr/lib64/libxml2.so.2.7.6) ==6573== by 0x4C79AE2: virXMLParseHelper (xml.c:779) ==6573== by 0x415512: cmdDomblklist (virsh.c:1726) ==6573== by 0x4149E2: vshCommandRun (virsh.c:16365) ==6573== by 0x427743: main (virsh.c:17867) ==6573== ==6573== LEAK SUMMARY: ==6573== definitely lost: 2,188 bytes in 2 blocks ==6573== indirectly lost: 45,657 bytes in 332 blocks ==6573== possibly lost: 0 bytes in 0 blocks ==6573== still reachable: 128,034 bytes in 1,364 blocks ==6573== suppressed: 0 bytes in 0 blocks 2011-12-09 Stefan Berger fix error when parsing ppc64 models on x86 host When parsing ppc64 models on an x86 host an out-of-memory error message is displayed due to it checking for retcpus being NULL. Fix this by removing the check whether retcpus is NULL since we will realloc into this variable. Also in the X86 model parser display the OOM error at the location where it happens. 2011-12-09 Stefan Berger fix memory leak in src/nodeinfo.c Fix memory leak: ==27534== 24 bytes in 1 blocks are definitely lost in loss record 207 of 530 ==27534== at 0x4A05E46: malloc (vg_replace_malloc.c:195) ==27534== by 0x38EC26EC37: vasprintf (in /lib64/libc-2.13.so) ==27534== by 0x4E998E6: virVasprintf (util.c:1677) ==27534== by 0x4E999F1: virAsprintf (util.c:1695) ==27534== by 0x4F1EAAC: nodeGetInfo (nodeinfo.c:593) ==27534== by 0x47948F: qemuCapsInitCPU (qemu_capabilities.c:855) ==27534== by 0x4796B1: qemuCapsInit (qemu_capabilities.c:915) ==27534== by 0x456550: qemuCreateCapabilities (qemu_driver.c:245) ==27534== by 0x4578C4: qemudStartup (qemu_driver.c:580) ==27534== by 0x4F20886: virStateInitialize (libvirt.c:852) ==27534== by 0x420E55: daemonRunStateInit (libvirtd.c:1156) ==27534== by 0x4E94C56: virThreadHelper (threads-pthread.c:157) Mark this leaked variable as const char * when it is passed into another function. 2011-12-09 Michal Privoznik threadpool: Don't wait on condition if pool has no workers Pool creates new workers dynamically. However, it is possible for a pool to have no workers. If we want to free that pool, we don't want to wait on quit condition as it will never be signaled. 2011-12-09 Jiri Denemark bridge: Fix forward delay APIs Due to copy&paste error in c1df2c14b590b3d68b707aa4f3a570f95a6bc548, virNetDevBridge[SG]etSTPDelay APIs were accessing wrong file. 2011-12-09 Peter Krempa cpu: Add cpu flags supported by newest qemu Add support for newly supported Intel cpu features. Newly supported flags are: pclmuldq, dtes64, smx, fma, pdcm, movbe, xsave, osxsave and avx. This adds support for Intel's Sandy Bridge platform. virsh: return correct value from cmdDomIfGetLink Reported by Alex Jia . Function cmdDomIfGetLink did not set a success return value on success path. 2011-12-09 Stefan Berger Pass the VM's UUID into the nwfilter subsystem A preparatory patch for DHCP snooping where we want to be able to differentiate between a VM's interface using the tuple of . We assume that MAC addresses could possibly be re-used between different networks (VLANs) thus do not only want to rely on the MAC address to identify an interface. At the current 'final destination' in virNWFilterInstantiate I am leaving the vmuuid parameter as ATTRIBUTE_UNUSED until the DHCP snooping patches arrive. (we may not post the DHCP snooping patches for 0.9.9, though) Mostly this is a pretty trivial patch. On the lowest layers, in lxc_driver and uml_conf, I am passing the virDomainDefPtr around until I am passing only the VM's uuid into the NWFilter calls. 2011-12-09 Stefan Berger nwfilter: cleanup return codes in nwfilter subsystem This patch cleans up return codes in the nwfilter subsystem. Some functions in nwfilter_conf.c (validators and formatters) are keeping their bool return for now and I am converting their return code to true/false. All other functions now have failure return codes of -1 and success of 0. [I searched for all occurences of ' 1;' and checked all 'if ' and adapted where needed. After that I did a grep for 'NWFilter' in the source tree.] 2011-12-09 Alex Jia virsh: plug memory leak on cmdDomIfGetLink() sucessful path Detected by valgrind. Leak introduced in commit dc675f3: * tools/virsh.c: fix memory leak on cmdDomIfGetLink. * how to reproduce? % valgrind -v --leak-check=full virsh domif-getlink 0 * actual valgrind result: ==13102== 18 bytes in 1 blocks are definitely lost in loss record 9 of 47 ==13102== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==13102== by 0x322A6A67DD: xmlStrndup (in /usr/lib64/libxml2.so.2.7.6) ==13102== by 0x414892: cmdDomIfGetLink (virsh.c:1538) ==13102== by 0x4136A2: vshCommandRun (virsh.c:16363) ==13102== by 0x4253FB: main (virsh.c:17865) ==13102== ==13102== LEAK SUMMARY: ==13102== definitely lost: 18 bytes in 1 blocks ==13102== indirectly lost: 0 bytes in 0 blocks ==13102== possibly lost: 0 bytes in 0 blocks ==13102== still reachable: 127,888 bytes in 1,361 blocks ==13102== suppressed: 0 bytes in 0 blocks 2011-12-09 Alex Jia virsh: plug memory leak on cmdBlkdeviotune() sucessful path Detected by valgrind. Leak introduced in commit e9bd9a0: * tools/virsh.c: fix memory leak on cmdBlkdeviotune. * how to reproduce? % valgrind -v --leak-check=full virsh blkdeviotune * actual valgrind result: ==12759== 576 bytes in 1 blocks are definitely lost in loss record 18 of 29 ==12759== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==12759== by 0x42134E: _vshCalloc.clone.2 (virsh.c:422) ==12759== by 0x4217CB: cmdBlkdeviotune (virsh.c:6364) ==12759== by 0x4136A2: vshCommandRun (virsh.c:16363) ==12759== by 0x4253FB: main (virsh.c:17865) ==12759== ==12759== LEAK SUMMARY: ==12759== definitely lost: 576 bytes in 1 blocks ==12759== indirectly lost: 0 bytes in 0 blocks ==12759== possibly lost: 0 bytes in 0 blocks ==12759== still reachable: 126,964 bytes in 1,342 blocks ==12759== suppressed: 0 bytes in 0 blocks 2011-12-08 Eric Blake maint: allow bootstrap in a sandbox Jiri Denemark reported an instance of bootstrapping libvirt failing when run inside a sandbox, traced to rpm trying to access /var/ which was not permitted by the sandbox. Alex Jia reported that 0.9.8-rc1 failed to bootstrap if patch(1) is not installed. * bootstrap.conf (buildreq): Avoid rpm call if python-config exists. Also, require patch, in case we have gnulib-local diffs. 2011-12-08 Laine Stump test: fix potential lock corruption in test driver In some error situations, the function testDomainRestoreFlags() could unlock the test driver mutex without first locking it. This patch moves the lock operation earlier, so that it occurs before any potential jump down to the unlock call. I found this problem while auditing the test driver lock usage to determine the cause of a hang while running the following test: cd tests; while true; do printf x; ./undefine; done This patch *does not* solve that problem, but we now understand its actual source, and danpb is working on a patch. 2011-12-08 Eric Blake spec: fix logic bug in deciding to turn on cgconfig https://bugzilla.redhat.com/show_bug.cgi?id=738725 Commit ecd8725 tried to silence a spurious warning on the initial libvirt install, and commit ba6cbb1 tried to fix up the logic to the correct Fedora version, but the warning was still present due to a logic bug: since %{fedora} and %{rhel} are never simulatanously set, then 0%{rhel} <= 6 made the %if always true. Checking for minimum versions (via >=) is okay, but checking for maximum versions (via <=) requires a prerequisite test that the platform being tested is non-zero. Also fix a bogus setting of with_libxl (although we previously hard-code with_libxl to 0 for rhel earlier in the file, so this was not as severe a bug). * libvirt.spec.in (with_cgconfig): Don't enable cgconfig on F16. 2011-12-08 Eric Blake spec: make it easier to autoreconf when building rpm Over time, Fedora and RHEL RPMs have often backported upstream patches that touched configure.ac and/or Makefile.am; this necessitates rerunning the autotools for the patch to be effective. Making this a one-liner spec tweak will make it easier for future backports to pull patches without having to find all the places to touch to properly use the autotools. Meanwhile, there have been historical instances where an update in the autotools caused FTBFS situations, so this is not on by default. * libvirt.spec.in (enable_autotools): New variable, default off. (BuildRequires): Conditionally add autotools. (%build): Conditionally use them before configure. * mingw32-libvirt.spec.in: Likewise. 2011-12-08 Daniel P. Berrange When checking nttyFDs to see if it is != 1, be sure to use '1' and not '-1' * src/lxc/lxc_controller.c: Fix check for tty count 2011-12-08 Daniel P. Berrange Fix installation of libvirt-guests.service The installation rules for the libvirt-guests.service were totally broken - Installing in the wrong location - The location was not overridable - The install-systemd rule was not invoked anywhere - The install-systemd rule was not invoking install-initscript which it depends on - The installed service file lacked a .service extension * tools/Makefile.am: Fix install of libvirt-guests.service 2011-12-08 Daniel P. Berrange Ensure to prefix %{buildroot} when overriding systemd install location The %makeinstall macro does not set DESTDIR, instead of explicitly prefixes %{buildroot} onto all paths. Thus we need to do the same when setting the systemd unit dir * libvirt.spec.in: Prefix %{buildroot} onto %{unitdir} 2011-12-08 Bharata B Rao Add ppc64 specific definitions to domain.rng ppc64 as new arch type and pseries as new machine type are added under ... . 2011-12-08 Prerna Saxena Clean up qemuBuildCommandLine to remove x86-specific assumptions from generic code. This implements the minimal set of changes needed in libvirt to launch a PowerPC-KVM based guest. It removes x86-specific assumptions about choice of serial driver backend from generic qemu guest commandline generation code. It also restricts the ACPI capability to be available for an x86 or x86_64 domain. This is not a complete solution -- it still does not guarantee libvirt the capability to flag non-supported options in guest XML. (Eg, an ACPI specification in a PowerPC guest XML will still get processed, even though qemu-system-ppc64 does not support it while qemu-system-x86_64 does.) This drawback exists because libvirt falls back on qemu to query supported features, and qemu '-h' blindly lists all capabilities -- irrespective of whether they are available while emulating a given architecture or not. The long-term solution would be for qemu to list out capabilities based on architecture and platform -- so that libvirt can cleanly make out what devices are supported on an arch (say 'ppc64') and platform (say, 'mac99'). 2011-12-08 Prerna Saxena Add support for ppc64 qemu This enables libvirt to select the correct qemu binary (qemu-system-ppc64) for a guest vm based on arch 'ppc64'. Also, libvirt is enabled to correctly parse the list of supported PowerPC CPUs, generated by running 'qemu-system-ppc64 -cpu ?' Acked-by: Stefan Berger 2011-12-08 Prerna Saxena Modify the tests/nodeinfotest.c to use sysfs in addition to proc/cpuinfo This patch creates a new sysfs hierarchy under tests/nodeinfodata/linux-nodeinfo-sysfs-test-1. Output files and /proc/cpuinfo files are also respectively added for both x86 and ppc64. 2011-12-08 Prerna Saxena Use sysfs to gather host topology, in place of /proc/cpuinfo Libvirt at present depends on /proc/cpuinfo to gather host details such as CPUs, cores, threads, etc. This is an architecture- dependent approach. An alternative is to use 'Sysfs', which provides a platform-agnostic interface to parse host CPU topology. 2011-12-08 Christophe Fergeau maint: move my name to commiters Since I have commit rights on libvirt-glib, I can also push to libvirt, Eric Blake told to move my name up to committers to better reflect reality. 2011-12-08 Daniel Veillard Release of libvirt-0.9.8 * configure.ac docs/news.html.in libvirt.spec.in: updated for the release * po/*.po*: fetched localization update and regenerated 2011-12-08 Eric Blake spec: don't use chkconfig --list https://bugzilla.redhat.com/show_bug.cgi?id=694403 reports that the specfile is incorrectly checking for a running libvirt-guests service. For example, $ LC_ALL=es_ES chkconfig --list libvirt-guests libvirt-guests 0:desactivado 1:desactivado 2:desactivado 3:activo 4:activo 5:activo 6:desactivado will fail to find 5:on, even though it is active. But chkconfig already has a mode where you can silently use the exit status to check for an active service. * libvirt.spec.in (%post): Use simpler chkconfig options, to avoid issues with localization. 2011-12-08 Eric Blake build: fix build with older libxml2 On RHEL 5, with libxml2-2.6.26, the build failed with: virsh.c: In function 'vshNodeIsSuperset': virsh.c:11951: warning: implicit declaration of function 'xmlChildElementCount' (or if warnings aren't errors, a link failure later on). * src/util/xml.h (virXMLChildElementCount): New prototype. * src/util/xml.c (virXMLChildElementCount): New function. * src/libvirt_private.syms (xml.h): Export it. * tools/virsh.c (vshNodeIsSuperset): Use it. 2011-12-08 Daniel P. Berrange Fix updating of haveTheBuck in RPC client to be race-free When one thread passes the buck to another thread, it uses virCondSignal to wake up the target thread. The variable 'haveTheBuck' is not updated in a race-free manner when this occurs. The current thread sets it to false, and the woken up thread sets it to true. There is a window where a 3rd thread can come in and grab the buck. Even if this didn't lead to crashes & deadlocks, this would still result in unfairness in the buckpassing algorithm. A better solution is to *never* set haveTheBuck to false when we're passing the buck. Only set it to false when there is no further thread waiting for the buck. * src/rpc/virnetclient.c: Only set haveTheBuck to false if no thread is waiting 2011-12-08 Daniel P. Berrange Revert fd066925440ba48acc95d8f31b2c98b1cc9d582d Commit fd066925440ba48acc95d8f31b2c98b1cc9d582d tried to fix a race condition in commit fa9595003d043df9f2efe95521c00898cef27106 Author: Daniel P. Berrange Date: Fri Nov 11 15:28:41 2011 +0000 Explicitly track whether the buck is held in remote client Unfortunately there is a second race condition whereby the event loop can trigger due to incoming data to read. Revert this fix, so a complete fix for the problem can be cleanly applied * src/rpc/virnetclient.c: Revert fd066925440ba48acc95d8f31b2c98b1cc9d582d 2011-12-07 Jim Fehlig Prevent crash of libvirtd when attaching to existing qemu process With security_driver set to "none" in /etc/libvirt/qemu.conf, libvirtd would crash when attempted to attach to an existing qemu process. Only copy the security model if it actually exists. 2011-12-07 Christophe Fergeau Add documentation for Fix typo in virDomainResume API doc It's referring to virSuspendDomain instead of virDomainSuspend. 2011-12-07 Jiri Denemark qemu: Ignore shutdown event from destroyed domain During virDomainDestroy, QEMU may emit SHUTDOWN event as a response to SIGTERM and since domain object is still locked, the event is processed after the domain is destroyed. We need to ignore this event in such case to avoid changing domain state from shutoff to shutdown. 2011-12-07 Osier Yang npiv: Expose fabric_name outside This patch is to expose the fabric_name of fc_host class, which might be useful for users who wants to known which fabric the (v)HBA connects to. The patch also adds the missed capabilities' XML schema of scsi_host, (of course, with fabric_wwn added), and update the documents (docs/formatnode.html.in) 2011-12-07 Daniel P. Berrange Conditionalize daemonPath decl for Win32 which lacks UNIX sockets 2011-12-07 Daniel P. Berrange Improve error reporting when libvirtd is not installed Currently if you try to connect to a local libvirtd when libvirtd is not in $PATH, you'll get an error error: internal error invalid use of command API This is because remoteFindDaemonPath() returns NULL, which causes us to pass NULL into virNetSocketConnectUNIX which in turn causes us to pass NULL into virCommandNewArgList. Adding missing error checks improves this to error: internal error Unable to locate libvirtd daemon in $PATH * src/remote/remote_driver.c: Report error if libvirtd cannot be found * src/rpc/virnetsocket.c: Report error if caller requested spawning of daemon, but provided no binary path 2011-12-05 Eric Blake spec: fix sanlock dependency * libvirt.spec.in (with_sanlock): On RHEL, don't force sanlock on architectures where it isn't available. 2011-12-05 Eric Blake spec: add dmidecode as prereq https://bugzilla.redhat.com/show_bug.cgi?id=754909 complains that because libvirt didn't require dmidecode, that the logs are noisy and virConnectGetSysinfo needlessly fails. Even 'virt-what' requires dmidecode, so it's not that onerous of a dependency. We may be able to drop this in the future when we move to parsing sysfs data, but for now, listing the dependency will help matters. * libvirt.spec.in (Requires): Sort Requires before BuildRequires. Add dmidecode. 2011-12-05 Eric Blake build: reduce warnings from older gcc Older gcc warns (on every file!) that -Wabi and -Wdeprecated only make sense on C++ projects. Newer gcc accepts these warnings for C, but it is not clear that they can do anything useful, so it is easier to just drop the warnings altogether. * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Silence -Wabi and -Wdeprecated on older gcc. Reported by Peter Krempa. 2011-12-05 Jiri Denemark examples: Update event tests for shutdown event 2011-12-05 Daniel P. Berrange Fix incorrect symbols for virtime.h module breaking Mingw32 The Mingw32 linker highlighted that the symbols for virtime.h declared in libvirt_private.syms were incorrect * src/libvirt_private.syms: Fix virtime.h symbols 2011-12-05 Jiri Denemark qemu: Rework handling of shutdown event When QEMU guest finishes its shutdown sequence, qemu stops virtual CPUs and when started with -no-shutdown waits for us to kill it using SGITERM. Since QEMU is flushing its internal buffers, some time may pass before QEMU actually dies. We mistakenly used "paused" state (and events) for this which is quite confusing since users may see a domain going to pause while they expect it to shutdown. Since we already have "shutdown" state with "the domain is being shut down" semantics, we should use it for this state. However, the state didn't have a corresponding event so I created one and called its detail as VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED (guest OS finished its shutdown sequence) with the intent to add VIR_DOMAIN_EVENT_SHUTDOWN_STARTED in the future if we have a sufficiently capable guest agent that can notify us when guest OS starts to shutdown. 2011-12-05 Guido Günther remote_driver: don't fail if keepalive check fails Otherwise connections to older libvirt abort with: $ virsh -c qemu+ssh://host.example.com/system list error: invalid connection pointer in virDrvSupportsFeature error: failed to connect to the hypervisor Tested against 0.8.3 and 0.9.8-rc2. 2011-12-05 Jiri Denemark Add support for QEMU 1.0 2011-12-05 Peter Krempa python: Expose binding for virNodeGetMemoryStats() This patch adds binding for virNodeGetMemoryStats method of libvirtd. Return value is represented as a python dictionary mapping field names to values. python: Expose binding for virNodeGetCPUStats() This patch adds binding for virNodeGetCPUStats method of libvirtd. Return value is represented as a python dictionary mapping field names to values. 2011-12-04 Eric Blake maint: fix improper use of 'an' https://bugzilla.redhat.com/show_bug.cgi?id=648855 mentioned a misuse of 'an' where 'a' is proper; that has since been fixed, but a search found other problems (some were a spelling error for 'and', while most were fixed by 'a'). * daemon/stream.c: Fix grammar. * src/conf/domain_conf.c: Likewise. * src/conf/domain_event.c: Likewise. * src/esx/esx_driver.c: Likewise. * src/esx/esx_vi.c: Likewise. * src/rpc/virnetclient.c: Likewise. * src/rpc/virnetserverprogram.c: Likewise. * src/storage/storage_backend_fs.c: Likewise. * src/util/conf.c: Likewise. * src/util/dnsmasq.c: Likewise. * src/util/iptables.c: Likewise. * src/xen/xen_hypervisor.c: Likewise. * src/xen/xend_internal.c: Likewise. * src/xen/xs_internal.c: Likewise. * tools/virsh.c: Likewise. 2011-12-03 Eric Blake build: require more tools from maintainers We want our tarballs to be complete - this means that any generated file that gets shipped as part of the tarball so that ordinary users don't have to rebuild it must be something that the maintainer can generate. There have been various reports of random build failures when using libvirt.git instead of a tarball, and often it is due to missing a maintainer-specific tool to produce one of these generated files. This patch raises the bar for what you must have installed to build libvirt.git, but does not impact what you can get away with for building tarballs. Note: It still remains possible to do a successful 'make dist' without these tools, when starting from a release tarball. * bootstrap.conf (buildreq): Add tools that maintainers need for a successful 'make dist' from a fresh git checkout. 2011-12-03 Eric Blake command: handle empty buffer argument correctly virBufferContentAndReset (intentionally) returns NULL for a buffer with no content, but it is feasible to invoke a command with an explicit empty string. * src/util/command.c (virCommandAddEnvBuffer): Reject empty string. (virCommandAddArgBuffer): Allow explicit empty argument. * tests/commandtest.c (test9): Test it. * tests/commanddata/test9.log: Adjust. 2011-12-03 Eric Blake build: fix build on Cygwin The RPC fixups needed on Linux are also needed on cygwin, and worked without further tweaking to the list of fixups. Also, unlike BSD, Cygwin exports 'struct ifreq', but unlike Linux, Cygwin lacks the ioctls that we were using 'struct ifreq' to access. This patch allows compilation under cygwin. * src/rpc/genprotocol.pl: Also perform fixups on cygwin. * src/util/virnetdev.c (HAVE_STRUCT_IFREQ): Also require AF_PACKET definition. * src/util/virnetdevbridge.c (virNetDevSetupControlFull): Only compile if SIOCBRADDBR works. 2011-12-02 Eric Blake build: fix build at -O2 on rawhide I had previously tested commit 059d746 with -O intentionally omitted from my CFLAGS; but that means that I missed out on this warning from gcc 4.6.2 when optimizations are enabled: util/buf.c: In function 'virBufferGetIndent': util/buf.c:86:1: error: function might be candidate for attribute 'pure' [-Werror=suggest-attribute=pure] While it is probably a good idea to add the attributes and silence this warning, it's also invasive; 'make -k' found more than 75 such complaints. And it doesn't help that gcc 4.6.2 is still buggy (coreutils reported a case where gcc 4.6.2 incorrectly suggested marking a function pure that incremented a global variable; fixed in gcc 4.7). So the best fix for now is to disable the warning. It also doesn't help that I stumbled across another problem - gcc documents that -Wsuggest-attribute=pure only warns if you use -O, or if you use -fipa-pure-const. But in practice, when I omitted -O but added -fipa-pure-const, the warnings are fickle - I got warnings for simple compilation that disappeared when I also added -fPIC. And the way libtool compiles things is with -fPIC first, then without -fPIC but with errors sent to /dev/null - which meant that without disabling -Wsuggest-attribute=pure, I got a compile error with no message. :( See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10197 * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Silence -Wsuggest-attribute warnings for now. 2011-12-02 Serge Hallyn apparmor: allow tunnelled migrations. The pathname for the pipe for tunnelled migration is unresolvable. The libvirt apparmor driver therefore refuses access, causing migration to fail. If we can't resolve the path, the worst that can happen is that we should have given permission to the file but didn't. Otherwise (especially since this is a /proc/$$/fd/N file) the file is already open and libvirt won't be refused access by apparmor anyway. Also adjust virt-aa-helper to allow access to the *.tunnelmigrate.dest.name files. For more information, see https://launchpad.net/bugs/869553. 2011-12-02 Peter Krempa client: Check if other thread claims it has the buck before claiming it. Originaly, the code checked if another client is the queue and infered ownership of the buck from that. Commit fa9595003d043df9f2efe95521c008 added a separate variable to track the buck. That caused, that a new call might enter claiming it has the buck, while another thread was signalled to take the buck. This ends in two threads claiming they hold the buck and entering poll(). This happens due to a race on waking up threads on the client lock mutex. This caused multi-threaded clients to hang, most prominently visible and reproducible on python based clients, like virt-manager. This patch causes threads, that have been signalled to take the buck to re-check if buck is held by another thread. 2011-12-02 Stefan Berger Update of filters to handle multiple IP addresses With fragments borrowed from David Steven's previous submission and some further modifications: A set of modifications to filters to handle multiple IP addresses (and MAC addresses) per interface. Also: - enable DHCP traffic from VM to any DHCP server - will require an update to a libvirt-tck data file 2011-12-02 Eric Blake virsh: translate net-info help Reported by kato.tomoyuki@jp.fujitsu.com at https://bugzilla.redhat.com/show_bug.cgi?id=749564 * tools/virsh.c (info_network_info): Mark string for translation. 2011-12-02 Eric Blake maint: typo fixes Many of these were mentioned by Yuri Chornoivan in: https://bugzilla.redhat.com/show_bug.cgi?id=669506 * src/esx/esx_vi.c (esxVI_WaitForTaskCompletion): Fix spelling. * src/conf/netdev_vport_profile_conf.c (virNetDevVPortProfileParse): Likewise. * src/xen/xend_internal.c (xenDaemonDomainSetVcpusFlags): Likewise. * src/xen/xm_internal.c (xenXMDomainSetVcpusFlags): Likewise. * src/esx/esx_util.c (esxUtil_ResolveHostname): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendFileSystemBuild): Likewise. * daemon/libvirtd.conf: Likewise. * src/util/logging.c (virLogMessage): Likewise. * src/uml/uml_conf.c (umlBuildCommandLineNet): Likewise. * src/vmx/vmx.c (virVMXFormatEthernet): Likewise. 2011-12-01 Eric Blake build: update to latest gnulib * .gnulib: Update to latest, for improved 'make syntax-check' and compiler warnings. * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Re-silence -Wformat-nonliteral. * cfg.mk (_test_script_regex): Recognize our test scripts. * gnulib/local/lib/*.diff: Drop, now that gnulib has this. * tests/virsh-optparse: Fix use of compare. * tests/virsh-schedinfo: Likewise. 2011-12-01 Eric Blake build: properly skip tests I got this failure on FreeBSD: shunloadtest.c: In function 'main': shunloadtest.c:150: error: 'EXIT_AM_SKIP' undeclared (first use in this function) but inspection showed several other problems, all fixed here. * tests/domainsnapshotxml2xmltest.c [!WITH_QEMU]: Ensure EXIT_AM_SKIP is defined. * tests/esxutilstest.c [!WITH_ESX]: Likewise. * tests/openvzutilstest.c [!WITH_OPENVZ]: Likewise. * tests/qemuargv2xmltest.c [!WITH_QEMU]: Likewise. * tests/qemuhelptest.c [!WITH_QEMU]: Likewise. * tests/qemuxml2argvtest.c [!WITH_QEMU]: Likewise. * tests/qemuxml2xmltest.c [!WITH_QEMU]: Likewise. * tests/qemuxmlnstest.c [!WITH_QEMU]: Likewise. * tests/shunloadtest.c [!linux]: Likewise. * tests/vmx2xmltest.c [!WITH_VMX]: Likewise. * tests/xml2vmxtest.c [!WITH_VMX]: Likewise. 2011-12-01 Daniel P. Berrange Avoid crash in shunloadtest For unknown reasons, the shunloadtest will crash on Fedora 16 inside dlopen() (gdb) bt #0 0x00000000000050e6 in ?? () #1 0x00007ff61a77b9d5 in floor () from /lib64/libm.so.6 #2 0x00007ff61e522963 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ff61e5297e6 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ff61e52917a in _dl_open () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ff61e0f6f26 in dlopen_doit () from /lib64/libdl.so.2 #7 0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ff61e0f752f in _dlerror_run () from /lib64/libdl.so.2 #9 0x00007ff61e0f6fc1 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2 #10 0x0000000000400a15 in main (argc=, argv=) at shunloadtest.c:105 Changing from RTLD_NOW to RTLD_LAZY avoids this problem, but quite possibly does not fix the root cause. * shunloadtest.c: s/NOW/LAZY/ 2011-12-01 Daniel P. Berrange Fix build for platforms lacking struct ifreq This ought to fix the build if you have net/if.h but do not have struct ifreq * configure.ac: Check for struct ifreq in net/if.h * src/util/virnetdev.c: Conditionalize to avoid use of struct ifreq if it does not exist 2011-12-01 Eric Blake build: fix 'make dist' without dtrace probes.h can only be generated on Linux, and then only with dtrace installed. If it is part of the tarball, then either 'make dist' will fail if you don't have that setup, or we would have to start keeping probes.h in libvirt.git. Since we only need it to be generated when dtrace is in use, it's better to avoid shipping it in the first place, and avoid tracking it in git. Meanwhile, there is a build dependency - since the RPC code is generated, it can be built early; but when dtrace is enabled, we must ensure probes.h is built even earlier. Commit 1afcfbdd tried to fix this, but did so in a way that added probes.h into the tarball, and broke VPATH as well. Commit ecbca767 fixed VPATH, but didn't fix the more fundamental problem. This patch solves the issue by adding a dependency instead. Tested with 'make dist' in a clean VPATH builds, for both './configure --without-dtrace' and './configure --with-dtrace'; all configurations were able to correctly build a tarball, and the dtrace configuration no longer sticks probes.h in the tarball. * src/Makefile.am (REMOTE_DRIVER_GENERATED): Don't ship probes.h; rather, make it a dependency. 2011-12-01 Lei Li Fix a logic error for setting block I/O Fix a logic error, the initial value of ret = -1, if just set --config, it will goto endjob directly without doing its really job here. 2011-12-01 Daniel P. Berrange Don't use undocumented __isleap macro The glibc time.h header has an undocumented __isleap macro that we are using. Since it is undocumented & does not appear on any other OS, stop using it and just define the macro in libvirt code instead. * src/util/virtime.c: Remove __isleap usage 2011-12-01 Michal Privoznik virsh: Allow other escape characters for console Currently virsh supports only ^] as escape character for console. However, some users might want to use something else. This patch creates such ability by specifying '-e' switch on virsh command line. 2011-12-01 Peter Krempa bridge_driver: Don't define network if XML contains more IPv4 adreses. Only one IPv4 DHCP definition is supported. Originally the code checked for a multiple definition and returned an error, but the new domain definition was already added to networks. This patch moves the check before the newly defined network is added to active networks. *src/network/bridge_driver.c: networkDefine(): - move multiple IPv4 addresses check before definition is used. 2011-12-01 Eric Blake spec: mark directories in /var/run as ghosts We have several directories that are created on the fly, and which only contain state relevant to a running libvirtd process (all located in /var/run). Since the directories are created as needed, and make no sense without a running libvirtd, we want them deleted if libvirt is uninstalled. And in F15 and newer, /var/run is on tmpfs (forcing us to recreate on the fly); which means that someone trying to verify a complete rpm will fail if the directory does not currently exist because libvirtd has not been started since boot. The solution, then, is to mark the directories as %ghost, so that rpm knows that we own them and will clean it up if libvirt is uninstalled, but will no longer create the directory for us at install, nor complain at verify time if the directory does not exist. See https://bugzilla.redhat.com/show_bug.cgi?id=656611. * libvirt.spec.in (%files): Add %ghost to temporary directories that we don't install, but want cleaned up on libvirt removal. 2011-12-01 Jiri Denemark virsh: Fix possible deadlock when virsh is about to exit Not only was ctl->quit accessed without a mutex but unfortunately, virEventAddTimeout only interrupts the poll when event loop is running so the hack needs to add a timeout that will make next poll return immediately without blocking. 2011-12-01 Alex Jia util: Plug memory leak on virNetDevMacVLanCreateWithVPortProfile() error path Detected by Coverity. Leak introduced in commit 90074ec. 2011-12-01 Alex Jia util: Plug memory leak on virNetDevBridgeGet() sucessful path Detected by Coverity. Leak introduced in commit c1df2c1. Two bugs here: 1. memory leak on successful parse 2. failure to parse still returned success 2011-12-01 Alex Jia uml: Plug memory leak on umlStartVMDaemon() error path Detected by Coverity. Leak introduced in commit 8866eed. Two bugs here: 1. logfd wasn't closed on all return paths 2. if we failed to mark a domain autodestroy, then the domain was not made transient but we still returned success 2011-12-01 Alex Jia rpc: Plug memory leak on virNetClientSendInternal() error path Detected by Coverity. Leak introduced in commit 673adba. Two separate bugs here: 1. call was not freed on all error paths 2. virCondDestroy was called even if virCondInit failed 2011-11-30 Alex Jia conf: Plug memory leak on virDomainDefParseXML() error path Detected by Coverity. Leak introduced in commit 0873b68. qemu: Plug memory leak onqemuProcessWaitForMonitor() error path Detected by Coverity. Leak introduced in commit 109efd7. 2011-11-30 Prerna Saxena Add PPC cpu driver. To add support for running libvirt on PowerPC, a CPU driver for the PowerPC platform must be added. Most generic cpu driver routines such as CPU compare, decode, etc are based on CPUID comparison and are not relevant for non-x86 platforms. Here, we introduce stubs for relevant PowerPC routines invoked by libvirt. 2011-11-30 Hu Tao qemu: filter blkio 0-device-weight at two other places filter 0-device-weight when: - getting blkio parameters with --config - starting up a domain When testing with blkio, I found these issues: (dom is down) virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,500 virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,0 virsh blkiotune dom weight : 800 device_weight : /dev/sda,200,/dev/sdb,0 # issue 1: shows 0 device weight of /dev/sdb that may confuse user (continued) virsh start dom # issue 2: If /dev/sdb doesn't exist, libvirt refuses to bring the # dom up because it wants to set the device weight to 0 of a # non-existing device. Since 0 means no weight-limit, we really don't # have to set it. 2011-11-30 Eric Blake qemu: amend existing table of device weights Prior to this patch, for a running dom, the commands: $ virsh blkiotune dom --device-weights /dev/sda,502,/dev/sdb,498 $ virsh blkiotune dom --device-weights /dev/sda,503 $ virsh blkiotune dom weight : 500 device_weight : /dev/sda,503 claim that /dev/sdb no longer has a non-default weight, but directly querying cgroups says otherwise: $ cat /cgroup/blkio/libvirt/qemu/dom/blkio.weight_device 8:0 503 8:16 498 After this patch, an explicit 0 is required to remove a device path from the XML, and omitting a device path that was previously specified leaves that device path untouched in the XML, to match cgroups behavior. * src/qemu/qemu_driver.c (parseBlkioWeightDeviceStr): Rename... (qemuDomainParseDeviceWeightStr): ...and use correct type. (qemuDomainSetBlkioParameters): After parsing string, modify rather than replacing existing table. * tools/virsh.pod (blkiotune): Tweak wording. 2011-11-30 Eric Blake conf: reject duplicate paths in device weights The next patch will make it possible to have virDomainSetBlkioParameters leave device weights unchanged if they are not mentioned in the incoming string, but this only works if the list of block weights does not allow duplicate paths. Technically, a user can still confuse libvirt by passing alternate spellings that resolve to the same device, but it is not worth worrying about working around that kind of abuse. * src/conf/domain_conf.c (virDomainDefParseXML): Require unique paths. 2011-11-30 Hu Tao virsh: fix setting weight and device-weights at the same time When setting both blkio weight and device-weights at the same time, the weight is lost. Fix it. 2011-11-30 Lei Li Add tests for blkdeviotune Support virDomain{Set, Get}BlockIoTune in the python API Python support for both setting and getting block I/O throttle. Enable the blkdeviotune command in virsh Support virsh command blkdeviotune. Can set or query a block disk I/O throttle setting. Implement virDomain{Set, Get}BlockIoTune for the qemu driver Implement the block I/O throttle setting and getting support to qemu driver. Support block I/O throttle in XML Enable block I/O throttle for per-disk in XML, as the first per-disk IO tuning parameter. Add virDomain{Set, Get}BlockIoTune support to the remote driver Support Block I/O Throttle setting and query to remote driver. 2011-11-30 Daniel P. Berrange Fix leak build config file path * src/libvirt.c: Free user directory path 2011-11-30 Daniel P. Berrange Remove time APIs from src/util/util.h The virTimestamp and virTimeMs functions in src/util/util.h duplicate functionality from virtime.h, in a non-async signal safe manner. Remove them, and convert all code over to the new APIs. * src/util/util.c, src/util/util.h: Delete virTimeMs and virTimestamp * src/lxc/lxc_driver.c, src/qemu/qemu_domain.c, src/qemu/qemu_driver.c, src/qemu/qemu_migration.c, src/qemu/qemu_process.c, src/util/event_poll.c: Convert to use virtime APIs 2011-11-30 Daniel P. Berrange Make logging async signal safe wrt time stamp generation Use the new virTimeStringNowRaw() API for generating log timestamps in an async signal safe manner * src/util/logging.c: Use virTimeStringNowRaw 2011-11-30 Daniel P. Berrange Add internal APIs for dealing with time The logging APIs need to be able to generate formatted timestamps using only async signal safe functions. This rules out using gmtime/localtime/malloc/gettimeday(!) and much more. Introduce a new internal API which is async signal safe. virTimeMillisNowRaw replacement for gettimeofday. Uses clock_gettime where available, otherwise falls back to the unsafe gettimeofday virTimeFieldsNowRaw replacements for gmtime(), convert a timestamp virTimeFieldsThenRaw into a broken out set of fields. No localtime() replacement is provided, because converting to local time is not practical with only async signal safe APIs. virTimeStringNowRaw replacements for strftime() which print a timestamp virTimeStringThenRaw into a string, using a pre-determined format, with a fixed size buffer (VIR_TIME_STRING_BUFLEN) For each of these there is also a version without the Raw postfix which raises a full libvirt error. These versions are not async signal safe * src/Makefile.am, src/util/virtime.c, src/util/virtime.h: New files * src/libvirt_private.syms: New APis * configure.ac: Check for clock_gettime in -lrt * tests/virtimetest.c, tests/Makefile.am: Test new APIs 2011-11-30 Daniel P. Berrange Remove obsolete virGetPMCapabilities sym from private symbols file Fix the build on Mingw32 by removing the now obsolete virGetPMCapabilities symbol from the private exports file * src/libvirt_private.syms: Remove virGetPMCapabilities 2011-11-30 Daniel P. Berrange Don't mark suspend as active until we know it is running If suspend failed for some reason (e.g. too short duration) then subsequent attempts to trigger suspend were rejected because we had already marked a suspend as being in progress * src/util/virnodesuspend.c: Don't mark suspend as active until we've successfully triggered it 2011-11-30 Daniel P. Berrange Add suspend info to Xen, LXC and UML hypervisor capabilities * src/lxc/lxc_conf.c, src/uml/uml_conf.c, src/xen/xen_hypervisor.c: Initialize suspend capabilities * tests/xencapsdata/*xml: Add empty powermgmt capabilities 2011-11-30 Daniel P. Berrange Remove pointless strdup in node suspend code The command name for the suspend action does not need to be strdup'd. The constant string can be used directly. This also means the code can be trivially rearranged to make the switch clearer * src/util/virnodesuspend.c: Remove strdup of cmdString 2011-11-30 Daniel P. Berrange Do lazy init of host PM features To avoid probing the host power management features on any call to virInitialize, only initialize the mutex in virNodeSuspendInit. Do lazy load of the supported PM target mask when it is actually needed * src/util/virnodesuspend.c: Lazy init of supported features 2011-11-30 Daniel P. Berrange Remove powerMgmt_valid field from capabilities struct If we ensure that virNodeSuspendGetTargetMask always resets *bitmask to zero upon failure, there is no need for the powerMgmt_valid field. * src/util/virnodesuspend.c: Ensure *bitmask is zero upon failure * src/conf/capabilities.c, src/conf/capabilities.h: Remove powerMgmt_valid field * src/qemu/qemu_capabilities.c: Remove powerMgmt_valid 2011-11-30 Daniel P. Berrange Add export of node suspend capabilities APIs * src/libvirt_private.syms: Export virNodeSuspendSupportsTarget and virNodeSuspendGetTargetMask 2011-11-30 Daniel P. Berrange Move suspend capabilities APIs out of util.h into virnodesuspend.c The node suspend capabilities APIs should not have been put into util.[ch]. Instead move them into virnodesuspend.[ch] * src/util/util.c, src/util/util.h: Remove suspend capabilities APIs * src/util/virnodesuspend.c, src/util/virnodesuspend.h: Add suspend capabilities APIs * src/qemu/qemu_capabilities.c: Include virnodesuspend.h 2011-11-30 Daniel P. Berrange Rename suspend capabilities APIs Rename virGetPMCapabilities to virNodeSuspendGetTargetMask and virDiscoverHostPMFeature to virNodeSuspendSupportsTarget. * src/util/util.c, src/util/util.h: Rename APIs * src/qemu/qemu_capabilities.c, src/util/virnodesuspend.c: Adjust for new names 2011-11-30 Daniel P. Berrange Sanitize virDiscoverHostPMFeature to return a boolean Since virDiscoverHostPMFeature is just checking one feature, there is no reason for it to return a bitmask. Change it to return a boolean * src/util/util.c, src/util/util.h: Make virDiscoverHostPMFeature return a boolean 2011-11-30 Daniel P. Berrange Move the virHostPMCapability enum helpers into capabilities.c The virHostPMCapability enum helper was declared in util.h but implemented in capabilities.c, which is in a completely separate library at link time. Move the declaration into the capabilities.c file and rename it to match normal conventions * src/util/util.h: Remove virHostPMCapability enum decl * src/conf/capabilities.c: Add virCapsHostPMTarget enum 2011-11-30 Daniel P. Berrange Fix capabilities XML to use generic terms for suspend targets The capabilities XML uses the x86 specific terms 'S3', 'S4' and 'Hybrid-Syspend'. Switch it to use the same terminology as the API constants and virsh options, eg 'suspend_mem' 'suspend_disk' and 'suspend_hybrid' * docs/formatcaps.html.in, docs/schemas/capability.rng, src/conf/capabilities.c: Rename suspend constants 2011-11-30 Daniel P. Berrange Remove internal only virHostPMCapability enum The internal virHostPMCapability enum just duplicates the public virNodeSuspendTarget enum, but with different names. * src/util/util.c: Use VIR_NODE_SUSPEND_TARGET constants * src/util/util.h: Remove virHostPMCapability enum * src/conf/capabilities.c: Use VIR_NODE_SUSPEND_TARGET_LAST 2011-11-30 Daniel P. Berrange Fix values of PM target type constants The VIR_NODE_SUSPEND_TARGET constants are not flags, so they should just be assigned straightforward incrementing values. * include/libvirt/libvirt.h.in: Change VIR_NODE_SUSPEND_TARGET values * src/util/virnodesuspend.c: Fix suspend target checks 2011-11-30 Alex Jia util: avoid null deref on qcowXGetBackingStore Detected by Coverity. the only case is caller passes a NULL to 'format' variable, then taking 'if (format)' false branch, the function qcow2GetBackingStoreFormat will directly dereferences the NULL 'format' pointer variable. 2011-11-30 Alex Jia virsh: correct return value error Fix cmdDomblklist to return 'true' on success instead of '0' https://bugzilla.redhat.com/show_bug.cgi?id=758590 2011-11-30 Lei Li Add new API virDomain{Set, Get}BlockIoTune This patch add new pulic API virDomainSetBlockIoTune and virDomainGetBlockIoTune. 2011-11-29 Hu Tao blkiotune: add qemu support for blkiotune.device_weight Implement setting/getting per-device blkio weights in qemu, using the cgroups blkio.weight_device tunable. 2011-11-29 Hu Tao blkiotune: add interface for blkiotune.device_weight This adds per-device weights to . Note that the cgroups implementation only supports weights per block device, and not per-file within the device; hence this option must be global to the domain definition rather than tied to individual / entries: /path/to/block 1000 .. This patch also adds a parameter --device-weights to virsh command blkiotune for setting/getting blkiotune.weight_device for any hypervisor that supports it. All entries under are concatenated into a single string attribute under virDomain{Get,Set}BlkioParameters, named "device_weight". 2011-11-29 Eric Blake qemu: fix blkiotune --live --config Without this, 'virsh blkiotune --live --config --weight=n' only affected live. * src/qemu/qemu_driver.c (qemuDomainSetBlkioParameters): Allow setting both configurations at once. 2011-11-29 Eric Blake qemu, lxc: drop redundant checks After the previous patch, there are now some redundant checks. * src/qemu/qemu_driver.c (qemudDomainGetVcpuPinInfo) (qemuGetSchedulerParametersFlags): Drop checks now guaranteed by libvirt.c. * src/lxc/lxc_driver.c (lxcGetSchedulerParametersFlags): Likewise. 2011-11-29 Eric Blake API: prevent query of --live and --config at once Drivers were inconsistent when presented both --live and --config at once. For example, within qemu, getting memory parameters favored live, getting blkio tuning favored config, and getting scheduler parameters errored out. Also, some, but not all, attempts to mix flags on query were filtered at the virsh level. We shouldn't have to duplicate efforts in every client app, nor in every driver. So, it is simpler to just enforce that the two flags cannot both be used at once on query operations, which has precedent in libvirt.c, and which matches the documentation of virDomainModificationImpact. * src/libvirt.c (virDomainGetMemoryParameters) (virDomainGetBlkioParameters) (virDomainGetSchedulerParametersFlags, virDomainGetVcpuPinInfo): Borrow sanity checking from virDomainGetVcpusFlags. 2011-11-29 Eric Blake build: fix typo in last patch * src/remote_protocol-structs: Use correct RPC name. 2011-11-29 Osier Yang block_resize: Update test file for RPC Pushed under build failure rule. block_resize: Expose the new API to virsh 2011-11-29 Osier Yang block_resize: Implement qemu driver method It requires the domain is running, otherwise fails. Resize to a lower size is supported, but should be used with extreme caution. In order to prohibit the "size" overflowing after multiplied by 1024. We do checking in the codes. For QMP mode, the default units is Bytes, the passed size needs to be multiplied by 1024, however, for HMP mode, the default units is "Megabytes", the passed "size" needs to be divided by 1024 then. 2011-11-29 Osier Yang block_resize: Implement qemu monitor functions Implements functions for both HMP and QMP mode. For HMP mode, qemu uses "M" as the units by default, so the passed "sized" is divided by 1024. For QMP mode, qemu uses "Bytes" as the units by default, the passed "sized" is multiplied by 1024. All of the monitor functions return -1 on failure, 0 on success, or -2 if not supported. 2011-11-29 Osier Yang block_resize: Wire up the remote protocol 2011-11-29 Osier Yang block_resize: Define the new API The new API is named as "virDomainBlockResize", intending to add support for qemu monitor command "block_resize" (both HMP and QMP). Similar with APIs like "virDomainSetMemoryFlags", the units for argument "size" is kilobytes. 2011-11-29 Michal Privoznik bandwidth: Fix funky identation 2011-11-29 Jiri Denemark virsh: Fix typos in man page lxc: Fix suspend/resume with freezer cgroup 2011-11-29 Osier Yang examples: Correct the example command to use testnode.xml * s/-connection/-c/ * Removes the redundant '/'. * Add "absolute" so that it's more clear. Pushed under trivial rule. 2011-11-29 Srivatsa S. Bhat Add virsh command to initiate suspend on the host Add a new command 'nodesuspend' to perform a timed suspend on the host. 2011-11-29 Srivatsa S. Bhat Implement the core API to suspend/resume the host Add the core functions that implement the functionality of the API. Suspend is done by using an asynchronous mechanism so that we can return the status to the caller before the host gets suspended. This asynchronous operation is achieved by suspending the host in a separate thread of execution. However, returning the status to the caller is only best-effort, but not guaranteed. To resume the host, an RTC alarm is set up (based on how long we want to suspend) before suspending the host. When this alarm fires, the host gets woken up. Suspend-to-RAM operation on a host running Linux can take upto more than 20 seconds, depending on the load of the system. (Freezing of tasks, an operation preceding any suspend operation, is given up after a 20 second timeout). And Suspend-to-Disk can take even more time, considering the time required for compaction, creating the memory image and writing it to disk etc. So, we do not allow the user to specify a suspend duration of less than 60 seconds, to be on the safer side, since we don't want to prematurely declare failure when we only had to wait for some more time. 2011-11-29 Srivatsa S. Bhat Add the remote protocol implementation for virNodeSuspendForDuration Implement the remote protocol for virNodeSuspendForDuration() API. Add a public API to invoke suspend/resume on the host Implement the public definitions for the new API virNodeSuspendForDuration() which will be subsequently used to do a timed suspend on the host. Add 'Hybrid-Suspend' power management discovery for the host Some systems support a feature known as 'Hybrid-Suspend', apart from the usual system-wide sleep states such as Suspend-to-RAM (S3) or Suspend-to-Disk (S4). Add the functionality to discover this power management feature and export it in the capabilities XML under the tag. 2011-11-29 Jiri Denemark rpc: Really send non-blocking calls while waiting for another call When another thread was dispatching while we wanted to send a non-blocking call, we correctly queued the call and woke up the thread but the thread just threw the call away since it forgot to recheck if its socket was writable. 2011-11-28 Michal Privoznik virsh: Don't traverse childless nodes in vshNodeIsSuperset If both nodes do not have any children, we pass zero to virBitmapAlloc which returns NULL. In turn we report OOM error and return false (meaning nodes are different). This is not true. 2011-11-28 Christian Franke virnetsocket: pass XAUTORITY for ssh connection When spawning an ssh connection, the environment variables DISPLAY, SSH_ASKPASS, ... are passed. However XAUTHORITY, which is necessary if the .Xauthority is in a non default place, was not passed. 2011-11-28 Lorin Hochstein conf: make virt-xml-validate work with vbox domains virt-xml-validate fails when run on a domain XML file of type 'vbox'. For failing test case, see https://bugzilla.redhat.com/show_bug.cgi?id=757097 This patch updates the XML schema to accept all valid hypervisor types, as well as dropping hypervisor types that are not in use by the current code base. 2011-11-28 Michal Privoznik conf: Improve incorrect root element error messages When user pass wrong root element, it is not 'internal error' and we can give him hint what we are expecting. 2011-11-28 Daniel P. Berrange Refactor initial LXC mem tune / device ACL code To make lxcSetContainerResources smaller, pull the mem tune and device ACL setup code out into separate methods * src/lxc/lxc_controller.c: Introduce lxcSetContainerMemTune and lxcSetContainerDeviceACL 2011-11-28 Daniel P. Berrange Add support for blkio tuning of LXC containers * src/lxc/lxc_controller.c: Refactor setting of initial blkio tuning parameters * src/lxc/lxc_driver.c: Enable live change of blkio tuning Add support for CPU quota/period to LXC driver * src/lxc/lxc_driver.c: Support changing quota/period for LXC containers * src/lxc/lxc_controller.c: Set initial quota/period at startup 2011-11-28 Daniel P. Berrange Support CPU placement in LXC driver While LXC does not have the concept of VCPUS, so we can't do per-VCPU pCPU placement, we can support the VM level CPU placement. Todo this simply set the CPU affinity of the LXC controller at startup. All child processes will inherit this affinity. * src/lxc/lxc_controller.c: Set process affinity 2011-11-28 Daniel P. Berrange Support NUMA memory placement for LXC containers Use numactl to set NUMA memory placement for LXC containers * src/lxc/lxc_controller.c: Support NUMA memory placement 2011-11-25 Michal Privoznik storage: Refetch file status after open This partly reverts my previous patch f88de3eb. We need to get file status after open, as given path could have been symlink, so fstat() will operate on different file than lstat(). 2011-11-25 Paolo Bonzini util: fix thinko in runIO When aligning you need to clear the bits in the mask and leave the others aside. Likely this code has never run, and will never run. 2011-11-25 Michal Privoznik storage: Skip socket and fifo on pool-start If pool directory contains special files like FIFO or sockets we want to skip those on pool-start or pool-refresh otherwise open() will get an error. conf: Don't drop console definition on domain restart One of my latest patches 2e37bf42d28d8bb5d045b206587c64643c64d02a copy serial console definition. On domain shutdown we save this info into state XML. However, later on the daemon start we simply drop this info and since we are not re-reading qemu log, vm->def->consoles[0] does not get populated with copy. Therefore we need to avoid dropping console definition if it is just alias for serial console. 2011-11-24 Jiri Denemark Fix version numbers for isAlive and setKeepAlive driver APIs build: Properly generate and check virkeepaliveprotocol-structs This fixes make dist broken by recent keepalive series examples: Use virConnectOpenAuth in events-c qemu: Cancel p2p migration when connection breaks If a connection to destination host is lost during peer-to-peer migration (because keepalive protocol timed out), we won't be able to finish the migration and it doesn't make sense to wait for qemu to transmit all data. This patch automatically cancels such migration without waiting for virDomainAbortJob to be called. qemu: Add support for keepalive messages during p2p migration Add keepalive support into domain-events examples Implement virConnectIsAlive in all drivers Introduce virConnectIsAlive API This API can be used to check if the socket associated with virConnectPtr is still open or it was closed (probably because keepalive protocol timed out). If there the connection is local (i.e., no socket is associated with the connection, it is trivially always alive. Implement keepalive protocol in remote driver Add support for async close of client RPC socket Implement keepalive protocol in libvirt daemon virsh: Always run event loop Since virsh already implements event loop, it has to also run it. So far the event loop was only running during virsh console command. Introduce virConnectSetKeepAlive virConnectSetKeepAlive public API can be used by a client connecting to remote server to start using keepalive protocol. The API is handled directly by remote driver and not transmitted over the wire to the server. Implement common keepalive handling These APIs are used by both client and server RPC layer to handle processing of keepalive messages. Define keepalive protocol The keepalive program has two procedures: PING, and PONG. Both are used only in asynchronous messages and the sender doesn't wait for any reply. However, the party which receives PING messages is supposed to react by sending PONG message the other party, but no explicit binding between PING and PONG messages is made. For backward compatibility neither server nor client are allowed to send keepalive messages before checking that remote party supports them. rpc: Add some debug messages to virNetClient rpc: Fix handling of non-blocking calls that could not be sent When virNetClientIOEventLoop is called for a non-blocking call and not even a single byte can be sent from this call without blocking, we properly reported that to the caller which properly frees the call. But we never removed the call from a call queue. rpc: Fix a typo in virNetClientSendNonBlock documentation rpc: Pass the buck only to the first available thread 2011-11-23 Stefan Berger nwfilter: remove virConnectPtr from internal API calls Remove the virConnectPtr from the nwfilter's internal API calls as far as possible. 2011-11-23 Peter Krempa qemu: Avoid dereference of NULL pointer If something fails while initializing qemu job object in qemuDomainObjPrivateAlloc(), memory to the private pointer is freed, but after that, the pointer is still dereferenced, which may result in a segfault. * qemuDomainObjPrivateAlloc() - Don't dereference NULL pointer. 2011-11-23 Eric Blake qemu: fix a const-correctness issue Generally, functions which return malloc'd strings should be typed as 'char *', not 'const char *', to make it obvious that the caller is responsible to free things. free(const char *) fails to compile, and although we have a cast embedded in VIR_FREE to work around poor code that frees const char *, it's better to not rely on that hack. * src/qemu/qemu_driver.c (qemuDiskPathToAlias): Change return type. (qemuDomainBlockJobImpl): Update caller. 2011-11-23 Eric Blake API: prefer 'disk' over 'block' or 'path' Given that we can now handle the target's disk shorthand, in addition to an absolute path to the file or block device used on the host, the term 'disk' fits a bit better as the parameter name than 'path'. * include/libvirt/libvirt.h.in: Update some parameter names. * src/libvirt.c (virDomainBlockStats, virDomainBlockStatsFlags) (virDomainBlockPeek, virDomainGetBlockInfo, virDomainBlockJobAbort) (virDomainGetBlockJobInfo, virDomainBlockJobSetSpeed) (virDomainBlockPull): Likewise. 2011-11-23 Eric Blake blockstats: support lookup by path in blockstats Commit 89b6284f made it possible to pass either a source name or the target device to most API demanding a disk designation, but forgot to update the documentation. It also failed to update virDomainBlockStats to take both forms. This patch fixes both the documentation and the remaining function. Xen continues to use just device shorthand (that is, I did not implement path lookup there, since xen does not track a domain_conf to quickly tie a path back to the device shorthand). * src/libvirt.c (virDomainBlockStats, virDomainBlockStatsFlags) (virDomainGetBlockInfo, virDomainBlockPeek) (virDomainBlockJobAbort, virDomainGetBlockJobInfo) (virDomainBlockJobSetSpeed, virDomainBlockPull): Document acceptable disk naming conventions. * src/qemu/qemu_driver.c (qemuDomainBlockStats) (qemuDomainBlockStatsFlags): Allow lookup by source name. * src/test/test_driver.c (testDomainBlockStats): Likewise. 2011-11-23 Michal Privoznik nwfilter: Initialize virNWFilterAddIpAddrForIfname return variable Latest nwfilter patch ad6c67cf introduced uninitialized return value. This was spotted by 4.6.2 gcc. 2011-11-23 Daniel P. Berrange Fix disabling of virtual port profile code on old hosts The WITH_VIRTUALPORT macro is defined to 0 when disabled, not left undefined. So #if must be used instead of #ifdef * src/util/virnetdevvportprofile.c: s/#ifdef/#if/ 2011-11-23 Daniel P. Berrange Fix uninitialized variable in NWfilter IP learning code * src/nwfilter/nwfilter_learnipaddr.c: Initialize ret variable 2011-11-23 Michal Prívozník virsh: Increase device-detach intelligence Up to now users have to give a full XML description on input when device-detaching. If they omitted something it lead to unclear error messages (like generated MAC wasn't found, etc.). With this patch users can specify only those information which specify one device sufficiently precise. Remaining information is completed from domain. 2011-11-23 Stefan Berger Enable detection of multiple IP addresses In preparation of DHCP Snooping and the detection of multiple IP addresses per interface: The hash table that is used to collect the detected IP address of an interface can so far only handle one IP address per interface. With this patch we extend this to allow it to handle a list of IP addresses. Above changes the returned variable type of virNWFilterGetIpAddrForIfname() from char * to virNWFilterVarValuePtr; adapt all existing functions calling this function. 2011-11-22 Eli Qiao fix error message when using wrong URI alias When configuring a URI alias like this in 'libvirt.conf': uri_aliases = [ "jj#j=qemu+ssh://root@127.0.0.1/system", "sleet=qemu+ssh://root@sleet.cloud.example.com/system", ] virsh -c jj#j It will show this error message: 'no connection driver available for No connection for URI jj#j' Actually，we expect this message below： Malformed 'uri_aliases' config entry 'jj#j=qemu+ssh://root@127.0.0.1/system', aliases may only contain 'a-Z, 0-9, _, -' Give this patch to fix this error. 2011-11-22 Stefan Berger Pass additional parameter into applyDHCPOnly function In preparation for the DHCP Snooping code: Pass an additional parameter into the applyDHCPOnly function of the 'techdriver'. 2011-11-22 Stefan Berger nwfilter: use shell variable to invoke 'ip(6)tables' command Introduce a shell variable 'IBT' to invoke the ip(6)tables command. Tested with libvirt-tck. 2011-11-22 Stefan Berger nwfilter: use shell variable to invoke 'ebtables' command Introduce a shell variable 'EBT' to invoke the ebtables command. Hard-code the used ebtables table to '-t nat'. Tested with libvirt-tck. 2011-11-22 Stefan Berger Improve error reporting of failures to apply filtering rules Display the executed command and failure message if a command failed to execute. Add documentation for STP filtering support Add documentation for the STP filtering support. Describe the XML attributes that are supported. Add test cases for STP traffic filtering This patch adds a few test cases for the XML parsing of STP filtering nodes. Add support for STP filtering This patch adds support for filtering of STP (spanning tree protocol) traffic to the parser and makes us of the ebtables support for STP filtering. This code now enables the filtering of traffic in chains with prefix 'stp'. Add a 'mac' chain With hunks borrowed from one of David Steven's previous patches, we now add the capability of having a 'mac' chain which is useful to filter for multiple valid MAC addresses. 2011-11-22 Eric Blake docs: fix grammar of capabilities * docs/formatcaps.html.in: Avoid run-on sentence, wrap lines. 2011-11-22 Daniel P. Berrange Add strings.h include to capabilities.h for ffs() function prototype On Mingw32 the ffs() function was not declared due to missing header include * src/conf/capabilities.c: The ffs() function lives in strings.h 2011-11-22 Osier Yang build: Update AUTHORS Add Chang Liu to the AUTHORS' list. 2011-11-22 Chang Liu storage: Fallback to use lvchange first if lvremove fails virStorageBackendLogicalDeleteVol() could not remove the lv with error "could not remove open logical volume" sometimes. Generally it's caused by the volume is still active, even if lvremove tries to remove it with option "--force". This patch is to fix it by disbale the lv first using "lvchange -aln" and "lvremove -f" afterwards if the direct "lvremove -f" failed. 2011-11-22 Srivatsa S. Bhat Export KVM Host Power Management capabilities This patch exports KVM Host Power Management capabilities as XML so that higher-level systems management software can make use of these features available in the host. The script "pm-is-supported" (from pm-utils package) is run to discover if Suspend-to-RAM (S3) or Suspend-to-Disk (S4) is supported by the host. If either of them are supported, then a new tag "" is introduced in the XML under the tag. However in case the query to check for power management features succeeded, but the host does not support any such feature, then the XML will contain an empty tag. In the event that the PM query itself failed, the XML will not contain any "power_management" tag. To use this, new APIs could be implemented in libvirt to exploit power management features such as S3/S4. 2011-11-21 Eric Blake conf: don't modify cpu set string during parsing None of the callers cared if str was updated to point to the next byte after the parsed cpuset; simplifying this results in quite a few code simplifications. Additionally, virCPUDefParseXML was strdup()'ing a malloc()'d string; avoiding a memory copy resulted in less code. * src/conf/domain_conf.h (virDomainCpuSetParse): Alter signature. * src/conf/domain_conf.c (virDomainCpuSetParse): Don't modify str. (virDomainVcpuPinDefParseXML, virDomainDefParseXML): Adjust callers. * src/conf/cpu_conf.c (virCPUDefParseXML): Likewise. * src/xen/xend_internal.c (sexpr_to_xend_topology): Likewise. * src/xen/xm_internal.c (xenXMDomainPinVcpu): Likewise. * src/xenxs/xen_sxpr.c (xenParseSxpr): Likewise. * src/xenxs/xen_xm.c (xenParseXM): Likewise. 2011-11-21 Roopa Prabhu qemu: don't release network actual device twice For direct attach devices, in qemuBuildCommandLine, we seem to be freeing actual device on error path (with networkReleaseActualDevice). But the actual device is not deleted. qemuProcessStop eventually deletes the direct attach device and releases actual device. But by the time qemuProcessStop is called qemuBuildCommandLine has already freed actual device, leaving stray macvtap devices behind on error. So the simplest fix is to remove the networkReleaseActualDevice in qemuBuildCommandLine. This patch does just that. 2011-11-21 Michal Privoznik qemu: Copy console definition from serial Now, when we support multiple consoles per domain, the vm->def->console[0] can still remain an alias for vm->def->serial[0]; However, we need to copy it's source definition as well otherwise we'll regress on virDomainOpenConsole. 2011-11-21 Daniel P. Berrange Fix distribution of .syms files for previous commit * src/Makefile.am: Remove libvirt_bridge.syms and libvirt_macvtap.syms from EXTRA_DIST. Add libvirt_dbus.syms 2011-11-21 Daniel P. Berrange Fix build on Mingw32 wrt export of virNetServerGetDBusConn Mingw32 complains if you request export of a symbol which does not in fact exist. * src/libvirt_bridge.syms, src/libvirt_macvtap.syms: Delete obsolete files * src/libvirt_private.syms: Remove virNetServerGetDBusConn * src/libvirt_dbus.syms: Add virNetServerGetDBusConn 2011-11-21 Osier Yang storage: Skips backingStore of virtual snapshot lv lvs outputs "[$lvname_vorigin]" for the virtual snapshot lv (created with "--virtualsize"), and the original device pointed by "$lvname_vorigin" is just for lvm internal use, one should never use it. Per lvm's nameing rules, "[" is not valid as part of the vg/lv name. (man 8 lvm). VALID NAMES The following characters are valid for VG and LV names: a-z A-Z 0-9 + _ . - VG and LV names cannot begin with a hyphen. There are also various reserved names that are used internally by lvm that can not be used as LV or VG names. A VG cannot be called anything that exists in /dev/ at the time of creation, nor can it be called '.' or '..'. A LV cannot be called '.' '..' 'snapshot' or 'pvmove'. The LV name may also not con‐ tain the strings '_mlog' or '_mimage' So we can skip the set the lv's backingStore by checking if the name begins with a "[". 2011-11-19 Stefan Berger Add documentation for VLAN filtering support Add documentation for the VLAN filtering support. Describe the XML attributes that are supported. Add test cases for VLAN traffic filtering This patch adds a few test cases for the XML parsing of VLAN filtering nodes. Add support for VLAN filtering This patch adds support for filtering of VLAN (802.1Q) traffic to the parser and makes us of the ebtables support for VLAN filtering. This code now enables the filtering of traffic in chains with prefix 'vlan'. 2011-11-19 Jim Fehlig Don't copy sexpr node value that is an empty string Xen4.1 initializes some unspecified sexpr config items to an empty string, unlike previous Xen versions that would leave the item unset. E.g. the kernel item for an HVM guest (non-direct kernel boot): Xen4.0 and earlier ... (image (hvm (kernel ) ... Xen4.1 ... (image (hvm (kernel '') ... The empty string for kernel causes some grief in subsequent parsing where existence of specified kernel is checked, e.g. if (!def->os.kernel) ... This patch solves the problem in sexpr_node_copy() by not copying a node containing an empty string. 2011-11-18 Eric Blake tests: avoid xend ABRT crash report I installed the xen development packages on my non-Xen F16 machine in order to compile-test xen code and ensure we don't break things on that front, but being a non-xen machine, /usr/sbin/xend is obviously not running. Unfortunately, xen-4.1.2-1.fc16 has a bug where merely trying to probe xend status on a non-xen kernel causes xend to issue an ABRT crash report: https://bugzilla.redhat.com/show_bug.cgi?id=728696 Even though libvirt (correctly) skips the test, the xend crash report is unnecessary noise. Fix this by first filtering out non-xen kernels even before attempting to probe xend. The test still runs and passes on a RHEL 5 xen kernel after this patch. * tests/reconnect.c (mymain): Skip xend probe on non-xen kernel. * tests/statstest.c (mymain): Likewise. 2011-11-18 Eric Blake build: fix compile error with no macvtap Since commit 6ec8288a, compilation has failed on RHEL 5: util/virnetdevmacvlan.c:672: error: conflicting types for 'virNetDevMacVLanCreateWithVPortProfile' * src/util/virnetdevmacvlan.c (virNetDevMacVLanCreateWithVPortProfile): Add missing parameter. 2011-11-18 Hu Tao enable cgroup cpuset by default This prepares for subsequent patches which introduce dependence on cgroup cpuset. Enable cgroup cpuset by default so users don't have to modify configuration file before encountering a cpuset error. 2011-11-18 Eric Blake build: fix accidental POTFILES.in regression The original patch for commit 4789fb2 considered renaming a file, then backed out the name change, but forgot to back out the POTFILES.in change, resulting in 'make syntax-check' failure. 2011-11-18 Eric Blake tests: test recent hash addition Excercise the new hash API, to ensure we avoid regressions. * tests/hashtest.c (testHashGetItems): New test. 2011-11-18 Stefan Berger Add test cases for parsing of list values This patch adds test cases for parsing of parameters with multiple occurrances of the same name. 2011-11-18 Stefan Berger Extend NWFilter parameter parser to cope with lists of values This patch modifies the NWFilter parameter parser to support multiple elements with the same name and to internally build a list of items. An example of the XML looks like this: The list of values is then stored in the newly introduced data type virNWFilterVarValue. The XML formatter is also adapted to print out all items in alphabetical order sorted by 'name'. This patch also fixes a bug in the XML schema on the way. 2011-11-18 Stefan Berger Create rules for each member of a list This patch extends the NWFilter driver for Linux (ebiptables) to create rules for each member of a previously introduced list. If for example an attribute value (internally) looks like this: IP = [10.0.0.1, 10.0.0.2, 10.0.0.3] then 3 rules will be generated for a rule accessing the variable 'IP', one for each member of the list. The effect of this is that this now allows for filtering for multiple values in one field. This can then be used to support for filtering/allowing of multiple IP addresses per interface. An iterator is introduced that extracts each member of a list and puts it into a hash table which then is passed to the function creating a rule. For the above example the iterator would cause 3 loops. 2011-11-18 Stefan Berger Rework value part of name-value pairs NWFilters can be provided name-value pairs using the following XML notation: The internal representation currently is so that a name is stored as a string and the value as well. This patch now addresses the value part of it and introduces a data structure for storing a value either as a simple value or as an array for later support of lists. This patch adjusts all code that was handling the values in hash tables and makes it use the new data type. 2011-11-18 Stefan Berger Documentation about chains' priorities, lists of elements etc. This patch adds several aspects of documentation about the network filtering system: - chains, chains' priorities and chains' default priorities - talks about lists of elements, i.e., a variable assigned multiple values (part of already ACK-ed series) - already mentions the vlan, stp and mac chains added later on (https://www.redhat.com/archives/libvir-list/2011-October/msg01238.html) - mentions limitations of vlan filtering (when sent by VM) on Linux systems 2011-11-18 Stefan Berger Add test cases Add test case for the chain names with known prefixes and the chain priority. 2011-11-18 Stefan Berger Interleave jumping into chains with filtering rules in 'root' table The previous patch extends the priority of filtering rules into negative numbers. We now use this possibility to interleave the jumping into chains with filtering rules to for example create the 'root' table of an interface with the following sequence of rules: Bridge chain: libvirt-I-vnet0, entries: 6, policy: ACCEPT -p IPv4 -j I-vnet0-ipv4 -p ARP -j I-vnet0-arp -p ARP -j ACCEPT -p 0x8035 -j I-vnet0-rarp -p 0x835 -j ACCEPT -j DROP The '-p ARP -j ACCEPT' rule now appears between the jumps. Since the 'arp' chain has been assigned priority -700 and the 'rarp' chain -600, the above ordering can now be achieved with the following rule: This patch now sorts the commands generating the above shown jumps into chains and interleaves their execution with those for generating rules. 2011-11-18 Stefan Berger Extend rule priorities into negative numbers So far rules' priorities have only been valid in the range [0,1000]. Now I am extending their priority into the range [-1000, 1000] for subsequently being able to sort rules and the access of (jumps into) chains following priorities. 2011-11-18 Stefan Berger Enable chains with names having a known prefix This patch enables chains that have a known prefix in their name. Known prefixes are: 'ipv4', 'ipv6', 'arp', 'rarp'. All prefixes are also protocols that can be evaluated on the ebtables level. Following the prefix they will be automatically connected to an interface's 'root' chain and jumped into following the protocol they evaluate, i.e., a table 'arp-xyz' will be accessed from the root table using ebtables -t nat -A -p arp -j I--arp-xyz thus generating a 'root' chain like this one here: Bridge chain: libvirt-O-vnet0, entries: 5, policy: ACCEPT -p IPv4 -j O-vnet0-ipv4 -p ARP -j O-vnet0-arp -p 0x8035 -j O-vnet0-rarp -p ARP -j O-vnet0-arp-xyz -j DROP where the chain 'arp-xyz' is accessed for filtering of ARP packets. 2011-11-18 Stefan Berger Extend the filter XML to support priorities of chains This patch extends the filter XML to support priorities of chains in the XML. An example would be: [...] The permitted values for priorities are [-1000, 1000]. By setting the priority of a chain the order in which it is accessed from the interface root chain can be influenced. 2011-11-18 Stefan Berger Use the actual names of chains in data structure Use the name of the chain rather than its type index (enum). This pushes the later enablement of chains with user-given names into the XML parser. For now we still only allow those names that are well known ('root', 'arp', 'rarp', 'ipv4' and 'ipv6'). 2011-11-18 Stefan Berger Use scripting for cleaning and renaming of chains Use scripts for the renaming and cleaning up of chains. This allows us to get rid of some of the code that is only capable of renaming and removing chains whose names are hardcoded. A shell function 'collect_chains' is introduced that is given the name of an ebtables chain and then recursively determines the names of all chains that are accessed from this chain and its sub-chains using 'jumps'. The resulting list of chain names is then used to delete all the found chains by first flushing and then deleting them. The same function is also used for renaming temporary filters to their final names. I tested this with the bash and dash as script interpreters. 2011-11-18 Stefan Berger Make filter creation in root table more flexible Use the previously introduced chain priorities to sort the chains for access from an interface's 'root' table and have them created in the proper order. This gets rid of a lot of code that was previously creating the chains in a more hardcoded way. To determine what protocol a filter is used for evaluation do prefix- matching, i.e., the filter 'arp' is used to filter for the 'arp' protocol, 'ipv4' for the 'ipv4' protocol and 'arp-xyz' will also be used to filter for the 'arp' protocol following the prefix 'arp' in its name. 2011-11-18 Stefan Berger Introduce an internal priority for chains For better handling of the sorting of chains introduce an internally used priority. Use a lookup table to store the priorities. For now their actual values do not matter just that the values cause the chains to be properly sorted through changes in the following patches. However, the values are chosen as negative so that once they are sorted along with filtering rules (whose priority may only be positive for now) they will always be instantiated before them (lower values cause instantiation before higher values). This is done to maintain backwards compatibility. Add function to get hash table's key/value pairs Add a function to the virHashTable for getting an array of the hash table's key-value pairs and have the keys (optionally) sorted. 2011-11-18 Daniel P. Berrange Add support for systemd init service This patch adds support for a systemd init service for libvirtd and libvirt-guests. The libvirtd.service is *not* written to use socket activation, since we want libvirtd to start on boot so it can do guest auto-start. The libvirt-guests.service is pretty lame, just exec'ing the original init script for now. Ideally we would factor out the functionality, into some shared tool. Instead of ./configure --with-init-script=redhat You can now do ./configure --with-init-script=systemd Or better still: ./configure --with-init-script=systemd+redhat We can also now support install of the upstart init script * configure.ac: Add systemd, and systemd+redhat options to --with-init-script option * daemon/Makefile.am: Install systemd services * daemon/libvirtd.sysconf: Add note about unused env variable with systemd * daemon/libvirtd.service.in: libvirtd systemd service unit * libvirt.spec.in: Add scripts to installing systemd services and migrating from legacy init scripts * tools/Makefile.am: Install systemd services * tools/libvirt-guests.init.sh: Rename to tools/libvirt-guests.init.in * tools/libvirt-guests.service.in: systemd service unit 2011-11-18 Daniel P. Berrange Add support for interfaces with type=direct to LXC Support creation of macvlan devices for LXC containers. Do not allow setting of bandwidth controls or vport profiles due to the complication that there is no host side visible device to work with. * src/lxc/lxc_driver.c: Support type=direct interfaces 2011-11-18 Daniel P. Berrange Allow creation of plain macvlan devices Update virNetDevMacVLanCreateWithVPortProfile to allow creation of plain macvlan devices, as well as macvtap devices. The former is useful for LXC containers * src/qemu/qemu_command.c: Explicitly request a macvtap device * src/util/virnetdevmacvlan.c, src/util/virnetdevmacvlan.h: Add new flag to allow switching between macvlan and macvtap creation 2011-11-18 Daniel P. Berrange Refactor LXC network setup to allow future enhancements The current lxcSetupInterfaces() method directly performs setup of the bridge devices. Since it will shortly need to also create macvlan devices, move the bridge related code into a separate method * src/lxc/lxc_driver.c: Split lxcSetupInterfaces() to create a new lxcSetupInterfaceBridge() 2011-11-18 Daniel P. Berrange Add missing 'const' annotations for internal domain conf helpers The virDomainNetGetActualBridgeName and virDomainNetGetActualDirectDev methods both return strings that point to data in the virDomainDefPtr struct, and should therefore not be freed. The return values should thus be 'const char *' not 'char *'. * src/conf/domain_conf.c, src/conf/domain_conf.h: Mark const * src/network/bridge_driver.c: Update to use a const char * 2011-11-18 Daniel P. Berrange Fix up ordering of private symbols file Fix ordering of symbols after re-arranging network device management API source files * src/libvirt_private.syms: Fix ordering 2011-11-18 Daniel P. Berrange Move ifaceMacvtapLinkDump and ifaceGetNthParent functions Move the ifaceMacvtapLinkDump and ifaceGetNthParent functions into virnetdevvportprofile.c since they are specific to that code. This avoids polluting the headers with the Linux specific netlink data types * src/util/interface.c, src/util/interface.h: Move ifaceMacvtapLinkDump and ifaceGetNthParent functions and delete remaining file * src/util/virnetdevvportprofile.c: Add ifaceMacvtapLinkDump and ifaceGetNthParent functions * src/network/bridge_driver.c, src/nwfilter/nwfilter_gentech_driver.c, src/nwfilter/nwfilter_learnipaddr.c, src/util/virnetdevmacvlan.c: Remove include of interface.h 2011-11-18 Daniel P. Berrange Move functions for dealing with physical/virtual devices Move virNetDevIsVirtualFunction, virNetDevGetVirtualFunctionIndex and virNetDevGetPhysicalFunction to virnetdev.c * src/util/interface.c, src/util/interface.h, src/util/virnetdev.c, src/util/virnetdev.h: Move APIs 2011-11-18 Daniel P. Berrange Rename APIs for dealing with virtual/physical functions Rename ifaceIsVirtualFunction to virNetDevIsVirtualFunction, ifaceGetVirtualFunctionIndex to virNetDevGetVirtualFunctionIndex and ifaceGetPhysicalFunction to virNetDevGetPhysicalFunction * src/util/interface.c, src/util/interface.h: Rename APIs * src/util/virnetdevvportprofile.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevValidateConfig to virnetdev.c * src/util/interface.c, src/util/interface.h: Remove virNetDevValidateConfig * src/util/virnetdev.c, src/util/virnetdev.h: Add virNetDevValidateConfig 2011-11-18 Daniel P. Berrange Rename ifaceCheck to virNetDevValidateConfig Rename the ifaceCheck method to virNetDevValidateConfig and change so that it always raises an error and returns -1 on error. * src/util/interface.c, src/util/interface.h: Rename ifaceCheck to virNetDevValidateConfig * src/nwfilter/nwfilter_gentech_driver.c, src/nwfilter/nwfilter_learnipaddr.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevGetIPv4Address to virnetdev.c Move the virNetDevGetIPv4Address function to virnetdev.c * util/interface.c, util/interface.h: Remove virNetDevGetIPv4Address * util/virnetdev.c, util/virnetdev.h: Add virNetDevGetIPv4Address 2011-11-18 Daniel P. Berrange Rename ifaceGetIPAddress to virNetDevGetIPv4Address To match up with the existing virNetDevSetIPv4Address, rename ifaceGetIPAddress to virNetDevGetIPv4Address * util/interface.h, util/interface.c: Rename API * network/bridge_driver.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevGetIndex & virNetDevGetVLanID to virnetdev.c Move virNetDevGetIndex & virNetDevGetVLanID to virnetdev.c to suit their functional purpose * util/interface.c, util/interface.h: Remove virNetDevGetIndex & virNetDevGetVLanID * util/virnetdev.c, util/virnetdev.h: Add virNetDevGetIndex & virNetDevGetVLanID 2011-11-18 Daniel P. Berrange Rename ifaceGetIndex and ifaceGetVLAN Rename the ifaceGetIndex method to virNetDevGetIndex and ifaceGetVlanID to virNetDevGetVLanID. Also change the error reporting behaviour to always raise errors and return -1 on failure * util/interface.c, util/interface.h: Rename ifaceGetIndex and ifaceGetVLAN * nwfilter/nwfilter_gentech_driver.c, nwfilter/nwfilter_learnipaddr.c, nwfilter/nwfilter_learnipaddr.c, util/virnetdevvportprofile.c: Update for API renames and error handling changes 2011-11-18 Daniel P. Berrange Move MAC address replacement functions to virnetdev.c Move virNetDevReplaceMacAddress and virNetDevRestoreMacAddress to the virnetdev.c file where they naturally belong * util/interface.c, util/interface.h: Remove virNetDevReplaceMacAddress and virNetDevRestoreMacAddress * util/virnetdev.c, util/virnetdev.h: Add virNetDevReplaceMacAddress and virNetDevRestoreMacAddress 2011-11-18 Daniel P. Berrange Rename interface MAC address replacement APIs Rename ifaceReplaceMacAddress to virNetDevReplaceMacAddress and ifaceRestoreMacAddress to virNetDevRestoreMacAddress. * util/interface.c, util/interface.h, util/virnetdevmacvlan.c: Rename APIs 2011-11-18 Daniel P. Berrange Move the low level macvlan creation APIs Move the low level macvlan creation APIs into the virnetdevmacvlan.c file where they more naturally belong * util/interface.c, util/interface.h: Remove virNetDevMacVLanCreate and virNetDevMacVLanDelete * util/virnetdevmacvlan.c, util/virnetdevmacvlan.h: Add virNetDevMacVLanCreate and virNetDevMacVLanDelete 2011-11-18 Daniel P. Berrange Rename low level macvlan creation APIs Rename ifaceMacvtapLinkAdd to virNetDevMacVLanCreate and ifaceLinkDel to virNetDevMacVLanDelete. Strictly speaking the latter isn't restricted to macvlan devices, but that's the only use libvirt has for it. * util/interface.c, util/interface.h, util/virnetdevmacvlan.c: Rename APIs 2011-11-18 Daniel P. Berrange Rename high level macvlan creation APIs Rename virNetDevMacVLanCreate to virNetDevMacVLanCreateWithVPortProfile and virNetDevMacVLanDelete to virNetDevMacVLanDeleteWithVPortProfile To make way for renaming the other macvlan creation APIs in interface.c * util/virnetdevmacvlan.c, util/virnetdevmacvlan.h, qemu/qemu_command.c, qemu/qemu_hotplug.c, qemu/qemu_process.c: Rename APIs 2011-11-18 Daniel P. Berrange Rename and split the macvtap.c file Rename the macvtap.c file to virnetdevmacvlan.c to reflect its functionality. Move the port profile association code out into virnetdevvportprofile.c. Make the APIs available unconditionally to callers * src/util/macvtap.h: rename to src/util/virnetdevmacvlan.h, * src/util/macvtap.c: rename to src/util/virnetdevmacvlan.c * src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h: Pull in vport association code * src/Makefile.am, src/conf/domain_conf.h, src/qemu/qemu_conf.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c: Update include paths & remove conditional compilation 2011-11-18 Daniel P. Berrange Rename Macvtap management APIs In preparation for code re-organization, rename the Macvtap management APIs to have the following patterns virNetDevMacVLanXXXXX - macvlan/macvtap interface management virNetDevVPortProfileXXXX - virtual port profile management * src/util/macvtap.c, src/util/macvtap.h: Rename APIs * src/conf/domain_conf.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_command.h, src/qemu/qemu_driver.c, src/qemu/qemu_hotplug.c, src/qemu/qemu_migration.c, src/qemu/qemu_process.c, src/qemu/qemu_process.h: Update for renamed APIs 2011-11-18 Daniel P. Berrange Fix use of uninitialized variable in QEMU driver 2011-11-17 Bharata B Rao qemu: Generate -numa option Add routines to generate -numa QEMU command line option based on ... XML specifications. 2011-11-17 Bharata B Rao XML definitions for guest NUMA and parsing routines This patch adds XML definitions for guest NUMA specification and contains routines to parse the same. The guest NUMA specification looks like this: ... ... 2011-11-17 Eric Blake snapshot: refuse to generate names for non-regular backing files For whatever reason, the kernel allows you to create a regular file named /dev/sdc.12345; although this file will disappear the next time devtmpfs is remounted. If you let libvirt generate the name of the external snapshot for a disk image originally using the block device /dev/sdc, then the domain will be rendered unbootable once the qcow2 file is lost on the next devtmpfs remount. In this case, the user should have used 'virsh snapshot-create --xmlfile' or 'virsh snapshot-create-as --diskspec' to specify the name for the qcow2 file in a sane location, rather than relying on libvirt generating a name that is most likely to be wrong. We can help avoid naive mistakes by enforcing that the user provide the external name for any backing file that is not a regular file. * src/conf/domain_conf.c (virDomainSnapshotAlignDisks): Only generate names if backing file exists as regular file. Reported by MATSUDA Daiki. 2011-11-16 Jim Fehlig Fix build with polkit0 I missed adding virNetServerGetDBusConn() to libvirtd_private.syms in commit b8adfcc6, which didn't cause a problem in 0.9.6 but results in this build error in 0.9.7 libvirtd-remote.o: In function `remoteDispatchAuthPolkit': remote.c:(.text+0x188dd): undefined reference to `virNetServerGetDBusConn' 2011-11-16 Jim Fehlig Revert commit 0f590c62 As noted by Daniel Berrange [1], the proper fix for the older PolicyKit build issue is to add virNetServerGetDBusConn to libvirt_private.syms. Revert unnecessary changes to daemon/Makefile.am [1] https://www.redhat.com/archives/libvir-list/2011-November/msg00852.html 2011-11-16 Laine Stump virsh: add iface-bridge and iface-unbridge commands One of the top questions by libvirt users is how to create a host bridge device so that guests can be directly on the physical network. There are several example documents that explain how to do this manually, but following them often results in confusion and failure. virt-manager does a good job of creating a bridge based on an existing network device, but not everyone wants to use virt-manager. This patch adds a new command, iface-bridge that makes it just about as simple as possible to create a new bridge device based on an existing ethernet/vlan/bond device (including associating IP configuration with the bridge rather than the now-attached device), and start that new bridge up ready for action, eg: virsh iface-bridge eth0 br0 For symmetry's sake, it also adds a command to remove a device from a bridge, restoring the IP config to the now-unattached device: virsh iface-unbridge br0 (I had a short debate about whether to do "iface-unbridge eth0" instead, but that would involve searching through all bridge devices for the one that contained eth0, which seems like a bit too much trouble). NOTE: These two commands require that the netcf library be available on the host. Hopefully this will provide some extra incentive for people using suse, debian, ubuntu, and other similar systems to polish up (and push downstream) the ports to those distros recently pushed to the upstream netcf repo by Dan Berrange. Anyone interested in helping with that effort in any way should join the netcf-devel mailing list (subscription info at https://fedorahosted.org/mailman/listinfo/netcf-devel) During creation of the bridge, it's possible to specify whether or not the STP protocol should be started up on the bridge and, if so, how many seconds the bridge should squelch traffic from newly added devices while learning new topology (defaults are stp='on' and delay='0', which seems to usually work best for bridges used in the context of libvirt guests). There is also an option to not immediately start the bridge (and a similar option to not immediately start the un-attached device after destroying the bridge. Default is to start the new device, because in the case of iface-unbridge not starting is strongly discouraged as it will leave the system with no network connectivity on that interface (because it's necessary to destroy/undefine the bridge device before the unattached device can be defined), and it seemed better to make the option for iface-bridge behave consistently. NOTE TO THOSE TRYING THESE COMMANDS FOR THE FIRST TIME: to guard against any "unexpected" change to configuration, it is advisable to issue an "virsh iface-begin" command before starting any interface config changes, and "virsh iface-commit" only after you've verified that everything is working as you expect. If something goes wrong, you can always run "virsh iface-rollback" or reboot the system (which should automatically do iface-rollback). Aside from adding the code for these two functions, and the two entries into the command table, the only other change to virsh.c was to add the option name to vshCommandOptInterfaceBy(), because the iface-unbridge command names its interface option as "bridge". virsh.pod has also been updated with short descriptions of these two new commands. 2011-11-16 Hu Tao fix a bug in remoteSerializeTypedParameters This is a fatal typo believed to be very likely to happen when using both i and j at the same time for indexing. 2011-11-16 Daniel P. Berrange Don't return a fatal error if receiving unexpected stream data Due to the asynchronous nature of streams, we might continue to receive some stream packets from the server even after we have shutdown the stream on the client side. These should be discarded silently, rather than raising an error in the RPC layer. * src/rpc/virnetclient.c: Discard stream data silently 2011-11-16 Daniel P. Berrange Fix handling of stream EOF Very occasionally the sequence of events from poll would result in getting a HANGUP on its own, instead of a HANGUP+READABLE at the same time. In the former case we would send back an error event to the client, but never send the empty packet to indicate EOF. 2011-11-16 Daniel P. Berrange Allow non-blocking message sending on virNetClient Add a new virNetClientSendNonBlock which returns 2 on full send, 1 on partial send, 0 on no send, -1 on error If a partial send occurs, then a subsequent call to any of the virNetClientSend* APIs will finish any outstanding I/O. TODO: the virNetClientEvent event handler could be used to speed up completion of partial sends if an event loop is present. * src/rpc/virnetsocket.h, src/rpc/virnetsocket.c: Add new virNetSocketHasPendingData() API to test for cached data pending send. * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add new virNetClientSendNonBlock() API to send non-blocking API 2011-11-16 Daniel P. Berrange Refactor code for enabling/disabling I/O callback in remote client * src/rpc/virnetclient.c: Add helper for setting I/O callback events 2011-11-16 Daniel P. Berrange Split virNetClientSend into 2 methods Stop multiplexing virNetClientSend for two different purposes, instead add virNetClientSendWithReply and virNetClientSendNoReply * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Replace virNetClientSend with virNetClientSendWithReply and virNetClientSendNoReply * src/rpc/virnetclientprogram.c, src/rpc/virnetclientstream.c: Update for new API names 2011-11-16 Daniel P. Berrange Refactor code for passing the buck in the remote client Remove some duplication by pulling the code for passing the buck out into a helper method * src/rpc/virnetclient.c: Introduce virNetClientIOEventLoopPassTheBuck 2011-11-16 Daniel P. Berrange Explicitly track whether the buck is held in remote client Instead of inferring whether the buck is held from the waitDispatch pointer, use an explicit 'bool haveTheBuck' field * src/rpc/virnetclient.c: Explicitly track the buck 2011-11-16 Daniel P. Berrange Remove all linked list handling from remote client event loop Directly messing around with the linked list is potentially dangerous. Introduce some helper APIs to deal with list manipulating the list * src/rpc/virnetclient.c: Create linked list handlers 2011-11-16 Eli Qiao util: Add netdev helper functions to private symbols 2011-11-16 Sage Weil qemu/rbd: improve rbd device specification This improves the support for qemu rbd devices by adding support for a few key features (e.g., authentication) and cleaning up the way in which rbd configuration options are passed to qemu. An member of the disk source xml specifies how librbd should authenticate. The username attribute is the Ceph/RBD user to authenticate as. The usage or uuid attributes specify which secret to use. Usage is an arbitrary identifier local to libvirt. The old RBD support relied on setting an environment variable to communicate information to qemu/librbd. Instead, pass those options explicitly to qemu. Update the qemu argument parsing and tests accordingly. 2011-11-15 Jim Fehlig Fix build with polkit0 I missed adding libvirt_driver_remote.la to libvirtd_LDADD in commit b8adfcc6, which didn't cause a problem in 0.9.6 but results in this build error in 0.9.7 libvirtd-remote.o: In function `remoteDispatchAuthPolkit': remote.c:(.text+0x188dd): undefined reference to `virNetServerGetDBusConn' 2011-11-15 Stefan Berger Fix strchr call triggering gcc 4.3 & 4.4 bug Replacing the strchr call with two variables through a strstr call. Calling strchr with two variables triggers a gcc 4.3/4.4 bug when used in combination with -Wlogical-op and at least -O1. maint: fix build include stdint.h to fix the build 2011-11-15 Daniel P. Berrange Remove ifaceSetMac and ifaceGetMac APIs The ifaceSetMac and ifaceGetMac APIs duplicate the functionality of the virNetDevSetMAC and virNetDevGetMAC APIs, but returning errno's instead of raising errors. * src/util/interface.c, src/util/interface.h: Remove ifaceSetMac and ifaceGetMac APIs, adjusting callers for new error behaviour 2011-11-15 Daniel P. Berrange Remove ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp APIs The ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp APIs can be replaced with calls to virNetDevSetOnline and virNetDevIsOnline * src/util/interface.c, src/util/interface.h: Delete ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp * src/nwfilter/nwfilter_gentech_driver.c, src/util/macvtap.c: Update to use virNetDevSetOnline and virNetDevIsOnline 2011-11-15 Daniel P. Berrange Move LXC veth.c code into shared utility APIs Move the virNetDevSetName and virNetDevSetNamespace APIs out of LXC's veth.c and into virnetdev.c. Move the remaining content of the file to src/util/virnetdevveth.c * src/lxc/veth.c: Rename to src/util/virnetdevveth.c * src/lxc/veth.h: Rename to src/util/virnetdevveth.h * src/util/virnetdev.c, src/util/virnetdev.h: Add virNetDevSetName and virNetDevSetNamespace * src/lxc/lxc_container.c, src/lxc/lxc_controller.c, src/lxc/lxc_driver.c: Update include paths 2011-11-15 Daniel P. Berrange Rename the LXC veth management APIs and delete duplicated APIs The src/lxc/veth.c file contains APIs for managing veth devices, but some of the APIs duplicate stuff from src/util/virnetdev.h. Delete thed duplicate APIs and rename the remaining ones to follow virNetDevVethXXXX * src/lxc/veth.c, src/lxc/veth.h: Rename APIs & delete duplicates * src/lxc/lxc_container.c, src/lxc/lxc_controller.c, src/lxc/lxc_driver.c: Update for API renaming 2011-11-15 Daniel P. Berrange Split src/util/network.{c,h} into 5 pieces The src/util/network.c file is a dumping ground for many different APIs. Split it up into 5 pieces, along functional lines - src/util/virnetdevbandwidth.c: virNetDevBandwidth type & helper APIs - src/util/virnetdevvportprofile.c: virNetDevVPortProfile type & helper APIs - src/util/virsocketaddr.c: virSocketAddr and APIs - src/conf/netdev_bandwidth_conf.c: XML parsing / formatting for virNetDevBandwidth - src/conf/netdev_vport_profile_conf.c: XML parsing / formatting for virNetDevVPortProfile * src/util/network.c, src/util/network.h: Split into 5 pieces * src/conf/netdev_bandwidth_conf.c, src/conf/netdev_bandwidth_conf.h, src/conf/netdev_vport_profile_conf.c, src/conf/netdev_vport_profile_conf.h, src/util/virnetdevbandwidth.c, src/util/virnetdevbandwidth.h, src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h, src/util/virsocketaddr.c, src/util/virsocketaddr.h: New pieces * daemon/libvirtd.h, daemon/remote.c, src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/conf/nwfilter_conf.h, src/esx/esx_util.h, src/network/bridge_driver.c, src/qemu/qemu_conf.c, src/rpc/virnetsocket.c, src/rpc/virnetsocket.h, src/util/dnsmasq.h, src/util/interface.h, src/util/iptables.h, src/util/macvtap.c, src/util/macvtap.h, src/util/virnetdev.h, src/util/virnetdevtap.c, tools/virsh.c: Update include files 2011-11-15 Daniel P. Berrange Fix error reporting in port profile parsing/formatting APIs The virtual port profile parsing/formatting APIs do not correctly handle unknown profile type strings/numbers. They behave as a no-op, instead of raising an error * src/util/network.c, src/util/network.h: Fix error handling of port profile APIs * src/conf/domain_conf.c, src/conf/network_conf.c: Update for API changes 2011-11-15 Daniel P. Berrange Rename virVirtualPortProfileParams & APIs Rename the virVirtualPortProfileParams struct to be virNetDevVPortProfile, and rename the APIs to match this prefix. * src/util/network.c, src/util/network.h: Rename port profile APIs * src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/network/bridge_driver.c, src/qemu/qemu_hotplug.c, src/util/macvtap.c, src/util/macvtap.h: Update for renamed APIs/structs 2011-11-14 Stefan Berger maint: Add Michael Wood as an author Add Michael Wood as an author. 2011-11-14 Michael Wood PATCH: Fix build without MACVTAP Hi Commit c31d23a78715f1144c73862c46ab0436de8b5e85 removed the "conn" parameter from qemuPhysIfaceConnect(), but it's still used if WITH_MACVTAP is false. Also, it's still mentioned in the comment above the function: /** * qemuPhysIfaceConnect: * @def: the definition of the VM (needed by 802.1Qbh and audit) * @conn: pointer to virConnect object * @driver: pointer to the qemud_driver * @net: pointer to he VM's interface description with direct device type * @qemuCaps: flags for qemu * * Returns a filedescriptor on success or -1 in case of error. */ int qemuPhysIfaceConnect(virDomainDefPtr def, struct qemud_driver *driver, virDomainNetDefPtr net, virBitmapPtr qemuCaps, enum virVMOperationType vmop) { int rc; #if WITH_MACVTAP [...] #else (void)def; (void)conn; (void)net; (void)qemuCaps; (void)driver; (void)vmop; qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("No support for macvtap device")); rc = -1; #endif return rc; } -- Michael Wood From f4fc43b4111a4c099395c55902e497b8965e2b53 Mon Sep 17 00:00:00 2001 From: Michael Wood Date: Sat, 12 Nov 2011 13:37:53 +0200 Subject: [PATCH] Fix build without MACVTAP. 2011-11-14 Guido Günther storage: forbid rebuilding existing disk storage pools which would blow away all volumes. Honor VIR_STORAGE_POOL_BUILD_OVERWRITE to force a rebuild. This was caught by libvirt-tck's storage/110-disk-pool.t. 2011-11-12 Eric Blake API: add trivial qemu support for VIR_TYPED_PARAM_STRING Qemu will be the first driver to make use of a typed string in the next round of additions. Separate out the trivial addition. * src/qemu/qemu_driver.c (qemudSupportsFeature): Advertise feature. (qemuDomainGetBlkioParameters, qemuDomainGetMemoryParameters) (qemuGetSchedulerParametersFlags, qemudDomainBlockStatsFlags): Allow typed strings flag where trivially supported. 2011-11-12 Eric Blake API: remote support for VIR_TYPED_PARAM_STRING Send and receive string typed parameters across RPC. This also completes the back-compat mentioned in the previous patch - the only time we have an older client talking to a newer server is if RPC is in use, so filtering out strings during RPC prevents returning an unknown type to the older client. * src/remote/remote_protocol.x (remote_typed_param_value): Add another union value. * daemon/remote.c (remoteDeserializeTypedParameters): Handle strings on rpc. (remoteSerializeTypedParameters): Likewise; plus filter out strings when replying to older clients. Adjust callers. * src/remote/remote_driver.c (remoteFreeTypedParameters) (remoteSerializeTypedParameters) (remoteDeserializeTypedParameters): Handle strings on rpc. * src/rpc/gendispatch.pl: Properly clean up typed arrays. * src/remote_protocol-structs: Update. Based on an initial patch by Hu Tao, with feedback from Daniel P. Berrange. 2011-11-12 Eric Blake API: add VIR_TYPED_PARAM_STRING This allows strings to be transported between client and server in the context of name-type-value virTypedParameter functions. For compatibility, o new clients will not send strings to old servers, based on a feature check o new servers will not send strings to old clients without the flag VIR_TYPED_PARAM_STRING_OKAY; this will be enforced at the RPC layer in the next patch, so that drivers need not worry about it in general. The one exception is that virDomainGetSchedulerParameters lacks a flags argument, so it must not return a string; drivers that forward that function on to virDomainGetSchedulerParametersFlags will have to pay attention to the flag. o the flag VIR_TYPED_PARAM_STRING_OKAY is set automatically, based on a feature check (so far, no driver implements it), so clients do not have to worry about it Future patches can then enable the feature on a per-driver basis. This patch also ensures that drivers can blindly strdup() field names (previously, a malicious client could stuff 80 non-NUL bytes into field and cause a read overrun). * src/libvirt_internal.h (VIR_DRV_FEATURE_TYPED_PARAM_STRING): New driver feature. * src/libvirt.c (virTypedParameterValidateSet) (virTypedParameterSanitizeGet): New helper functions. (virDomainSetMemoryParameters, virDomainSetBlkioParameters) (virDomainSetSchedulerParameters) (virDomainSetSchedulerParametersFlags) (virDomainGetMemoryParameters, virDomainGetBlkioParameters) (virDomainGetSchedulerParameters) (virDomainGetSchedulerParametersFlags, virDomainBlockStatsFlags): Use them. * src/util/util.h (virTypedParameterArrayClear): New helper function. * src/util/util.c (virTypedParameterArrayClear): Implement it. * src/libvirt_private.syms (util.h): Export it. Based on an initial patch by Hu Tao, with feedback from Daniel P. Berrange. 2011-11-12 Eli Qiao util: fix compile error on debian Add virnetdev.h,virnetdevbridge.h,virnetdevtap.h to private symbols, since debian linker no longer allows transitive link resolution 2011-11-12 Eric Blake qemu: fix domjobabort regression This reverts commit ef1065cf5ac; see also this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=751900 In qemu 0.15.1 and earlier, during migration to file, the qemu_savevm_state_begin and qemu_savevm_state_iterate methods will both process as much migration data as possible until either 1. The file descriptor returns EAGAIN 2. The bandwidth rate limit is reached If we set the rate limit to ULONG_MAX, test 2 never becomes true. We're passing a plain file descriptor to QEMU and POSIX does not support EAGAIN on regular files / block devices, so test 1 never becomes true either. In the 'virsh save --bypass-cache' case, we pass a pipe instead of a regular fd, but using a pipe adds I/O overhead, so always passing a pipe just so qemu can see EAGAIN doesn't seem nice. The ultimate fix needs to come from qemu - background migration must respect asynchronous abort requests, or else periodically return control to the main handling loop without an EAGAIN and without waiting to hit an insanely large amount of data. But until a version of qemu is fixed to support "unlimited" data rates while still allowing cancellation, the best we can do is avoid the automatic use of unlimited rates from within libvirt (users can still explicitly change the migration rates, if they are aware that they are giving up the ability to cancel a job). Reverting the lone use of QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX is the simplest patch; this slows migration back down to a default 32M/sec cap, but also ensures that the main qemu processing loop will still be responsive to cancellation requests. Hopefully upstream qemu will provide us a means of safely using unlimited speed, including a runtime probe of that capability. * src/qemu/qemu_migration.c (qemuMigrationToFile): Revert attempt to use unlimited migration bandwidth when migrating to file. 2011-11-12 Hu Tao fix two bugs in bridge_driver.c steps to reproduce: 1. having a network xml file(named default.xml) like this one: default c5322c4c-81d0-4985-a363-ad6389780d89 in /etc/libvirt/qemu/networks/, and mark it as autostart: $ ls -l /etc/libvirt/qemu/networks/autostart total 0 lrwxrwxrwx 1 root root 14 Oct 12 14:02 default.xml -> ../default.xml 2. start libvirtd and the device virbr0 is not automatically up. The reason is that the function virNetDevExists is now returns 1 if the device exists, comparing to the former one returns 0 if the device exists. But with only this fix will cause a segmentation fault(the same steps as above) that is fixed by the second chunk of code. 2011-11-11 Eric Blake build: drop useless dirent.h includes * .gnulib: Update to latest, for improved syntax-check. * src/lxc/lxc_container.c (includes): Drop unused include. * src/network/bridge_driver.c: Likewise. * src/node_device/node_device_linux_sysfs.c: Likewise. * src/openvz/openvz_driver.c: Likewise. * src/qemu/qemu_conf.c: Likewise. * src/storage/storage_backend_iscsi.c: Likewise. * src/storage/storage_backend_mpath.c: Likewise. * src/uml/uml_conf.c: Likewise. * src/uml/uml_driver.c: Likewise. 2011-11-11 Eric Blake xenapi: remove unused variable CC libvirt_driver_xenapi_la-xenapi_driver.lo xenapi/xenapi_driver.c: In function 'xenapiDomainGetVcpus': xenapi/xenapi_driver.c:1209:21: error: variable 'cpus' set but not used [-Werror=unused-but-set-variable] * src/xenapi/xenapi_driver.c (xenapiDomainGetVcpus): Silence compiler warning. 2011-11-11 Eric Blake maint: use mailmap, not AUTHORS, for secondary addresses * AUTHORS: Move Stefan's second entry... * .mailmap: ...here. 2011-11-10 Stefan Berger maint: fix make syntax-check Add my 2nd email to the list of AUTHORS to get 'make syntax-check' to pass. 2011-11-10 Stefan Berger Remove code instantiating filters on direct interfaces Remove the code that instantiates network filters on direct type of interfaces. The parser already does not accept it. 2011-11-10 Daniel P. Berrange Disable numactl on ARM architectures too * libvirt.spec.in: Disable numactl on ARM Add libvirt confdir to files section in mingw32 spec * mingw32-libvirt.spec.in: Ensure we own the confdir 2011-11-10 Eric Blake nwfilter: simplify execution of ebiptables scripts It's not worth even worrying about a temporary file, unless we ever expect the script to exceed maximum command-line argument length limits. * src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI): Run the commands as an argument to /bin/sh, rather than worrying about a temporary file. (ebiptablesWriteToTempFile): Delete unused function. 2011-11-10 Eric Blake nwfilter: avoid failure with noexec /tmp If /tmp is mounted with the noexec flag (common on security-conscious systems), then nwfilter will fail to initialize, because we cannot run any temporary script via virRun("/tmp/script"); but we _can_ use "/bin/sh /tmp/script". For that matter, using /tmp risks collisions with other unrelated programs; we already have /var/run/libvirt as a dedicated temporary directory for use by libvirt. * src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesWriteToTempFile): Use internal directory, not /tmp; drop attempts to make script executable; and detect close error. (ebiptablesExecCLI): Switch to virCommand, and invoke the shell to read the script, rather than requiring an executable script. 2011-11-09 Daniel P. Berrange Adjust naming of network device bandwidth management APIs Rename virBandwidth to virNetDevBandwidth, and virRate to virNetDevBandwidthRate. * src/util/network.c, src/util/network.h: Rename bandwidth structs and APIs * src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/util/macvtap.c, src/util/macvtap.h, tools/virsh.c: Update for API changes. 2011-11-09 Daniel P. Berrange Santize naming of socket address APIs The socket address APIs in src/util/network.h either take the form virSocketAddrXXX, virSocketXXX or virSocketXXXAddr. Sanitize this so everything is virSocketAddrXXXX, and ensure that the virSocketAddr parameter is always the first one. * src/util/network.c, src/util/network.h: Santize socket address API naming * src/conf/domain_conf.c, src/conf/network_conf.c, src/conf/nwfilter_conf.c, src/network/bridge_driver.c, src/nwfilter/nwfilter_ebiptables_driver.c, src/nwfilter/nwfilter_learnipaddr.c, src/qemu/qemu_command.c, src/rpc/virnetsocket.c, src/util/dnsmasq.c, src/util/iptables.c, src/util/virnetdev.c, src/vbox/vbox_tmpl.c: Update for API renaming 2011-11-09 Daniel P. Berrange Split bridge.h into three separate files Following the renaming of the bridge management APIs, we can now split the source file into 3 corresponding pieces * src/util/virnetdev.c: APIs for any type of network interface * src/util/virnetdevbridge.c: APIs for bridge interfaces * src/util/virnetdevtap.c: APIs for TAP interfaces * src/util/virnetdev.c, src/util/virnetdev.h, src/util/virnetdevbridge.c, src/util/virnetdevbridge.h, src/util/virnetdevtap.c, src/util/virnetdevtap.h: Copied from bridge.{c,h} * src/util/bridge.c, src/util/bridge.h: Split into 3 pieces * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/openvz/openvz_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_conf.h, src/uml/uml_conf.c, src/uml/uml_conf.h, src/uml/uml_driver.c: Update #include directives 2011-11-09 Daniel P. Berrange Remove usage of brctl command line tool Convert the virNetDevBridgeSetSTP and virNetDevBridgeSetSTPDelay to use ioctls instead of spawning brctl. Implement the virNetDevBridgeGetSTP and virNetDevBridgeGetSTPDelay methods which were declared in the header but never existed * src/util/bridge.c: Convert to use bridge ioctls instead of brctl 2011-11-09 Daniel P. Berrange Add an API for retrieving the MAC address of an interface * src/util/bridge.c, src/util/bridge.h: Add virNetDevGetMAC 2011-11-09 Daniel P. Berrange Expose MTU management APIs The MTU management APIs are useful to other code inside libvirt, so should be exposed as non-static APIs. * src/util/bridge.c, src/util/bridge.h: Expose virNetDevSetMTU, virNetDevSetMTUFromDevice & virNetDevGetMTU 2011-11-09 Daniel P. Berrange Turn two int parameters into bools in bridge APIs * src/util/bridge.c, src/util/bridge.h: s/int/bool/ in virNetDevSetOnline and virNetDevBridgeSetSTP 2011-11-09 Daniel P. Berrange Rename all brXXXX APIs to follow new convention The existing brXXX APIs in src/util/bridge.h are renamed to follow one of three different conventions - virNetDevXXX - operations for any type of interface - virNetDevBridgeXXX - operations for bridge interfaces - virNetDevTapXXX - operations for tap interfaces * src/util/bridge.h, src/util/bridge.c: Rename all APIs * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Update for API renaming 2011-11-09 Daniel P. Berrange Make all brXXX APIs raise errors, instead of returning errnos Currently every caller of the brXXX APIs has to store the returned errno value and then raise an error message. This results in inconsistent error messages across drivers, additional burden on the callers and makes the error reporting inaccurate since it is hard to distinguish different scenarios from 1 errno value. * src/util/bridge.c: Raise errors instead of returning errnos * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Remove error reporting code 2011-11-09 Daniel P. Berrange Remove 'brControl' object The bridge management APIs in src/util/bridge.c require a brControl object to be passed around. This holds the file descriptor for the control socket. This extra object complicates use of the API for only a minor efficiency gain, which is in turn entirely offset by the need to fork/exec the brctl command for STP configuration. This patch removes the 'brControl' object entirely, instead opening the control socket & closing it again within the scope of each method. The parameter names for the APIs are also made to consistently use 'brname' for bridge device name, and 'ifname' for an interface device name. Finally annotations are added for non-NULL parameters and return check validation * src/util/bridge.c, src/util/bridge.h: Remove brControl object and update API parameter names & annotations. * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/uml/uml_conf.h, src/uml/uml_conf.c, src/uml/uml_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c: Remove reference to 'brControl' object 2011-11-09 Eric Blake build: fix mingw build of gnulib openpty Commit f7bd00c12 pulled in a gnulib module that fails to compile on mingw. Work around it while waiting for an upstream gnulib fix. * gnulib/local/lib/pty.in.h (openpty): Provide forward declarations of opaque structs not present on mingw. * gnulib/local/lib/openpty.c (openpty): Provide stub for mingw. 2011-11-09 Eric Blake build: allow for local gnulib diffs Commit f7bd00c12 pulled in a gnulib module that fails to compile on mingw. While it would be nice to pull in a newer version of .gnulib that fixes this, it is difficult to backport any .gnulib update to older releases. So, it makes sense to take advantage of gnulib-tool's ability to support local diffs, where we can apply specific diffs in our use of gnulib without waiting for upstream gnulib to pick up those changes, as well as avoiding a wholesale .gnulib update. The existence of local diffs will also make it easier to backport fixes against a tarball (as long as a tarball and libvirt.git share the same .gnulib commit, then the tarball can be patched by applying the same local diffs as a post-release libvirt.git commit, without having to rerun an entire gnulib-tool bootstrap). This patch introduces the framework for supporting local diffs, without actually introducing any. * bootstrap.conf (local_gl_dir): New variable. * autogen.sh (bootstrap_hash): Hash any local diffs, to force a re-bootstrap if just diffs change. * cfg.mk (_update_required): Likewise. 2011-11-09 Alex Jia lxc: free error object to avoid memory leak Detected by Coverity. Leak introduced in commit 9d201a5. * src/lxc/lxc_driver.c: Clean up on failure. 2011-11-09 Alex Jia lxc: free 'ttyFDs' array on return from lxcVmStart Detected by Coverity. Leak introduced in commit 0f31f7b. * src/lxc/lxc_driver.c: Clean up on failure. 2011-11-09 Osier Yang qemu: Fix improper error message for disk detaching s/virDomainDeviceTypeToString/virDomainDiskDeviceTypeToString/ Report by Xu He Jie virsh: Add VSH_OFLAG_EMPTY_OK for attach-disk command As the description of removing CDROM media from http://wiki.libvirt.org/page/QEMUSwitchToLibvirt#eject_DEV Add flag 'VSH_OFLAG_EMPTY_OK' to the option 'source' of attach-disk Then avoid outputting in the XML if 'source' was empty, rather than trusting libvirt domain_conf.c to understand an empty string. 2011-11-08 Daniel Veillard Release of libvirt-0.9.7 * confiure.ac docs/news.html.in libvirt.spec.in: update for release * po/*.po*: update localizations and rebuilt 2011-11-07 Eric Blake build: fix build on platforms without ptsname_r MacOS lacks ptsname_r, and gnulib doesn't (yet) provide it. But we can avoid it altogether, by using gnulib openpty() instead. Note that we do _not_ want the pt_chown module; gnulib uses it only to implement a replacement openpty() if the system lacks both openpty() and granpt(), but all systems that we currently port to either have at least one of openpty() and/or grantpt(), or lack ptys altogether. That is, we aren't porting to any system that requires us to deal with the hassle of installing a setuid pt_chown helper just to use gnulib's ability to provide openpty() on obscure platforms. * .gnulib: Update to latest, for openpty fixes * bootstrap.conf (gnulib_modules): Add openpty, ttyname_r. (gnulib_tool_option_extras): Exclude pt_chown module. * src/util/util.c (virFileOpenTty): Rewrite in terms of openpty and ttyname_r. * src/util/util.h (virFileOpenTtyAt): Delete dead prototype. 2011-11-07 Daniel P. Berrange Add missing defaultConsoleTargetType callback for AppArmour Every instance of virCapsPtr must have the defaultConsoleTargetType field set. * src/security/virt-aa-helper.c: Add defaultConsoleTargetType to virCapsPtr 2011-11-07 Daniel P. Berrange Fix sending/receiving of FDs when stream returns EAGAIN The code calling sendfd/recvfd was mistakenly assuming those calls would never block. They can in fact return EAGAIN and this is causing us to drop the client connection when blocking ocurrs while sending/receiving FDs. Fixing this is a little hairy on the incoming side, since at the point where we see the EAGAIN, we already thought we had finished receiving all data for the packet. So we play a little trick to reset bufferOffset again and go back into polling for more data. * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Update virNetSocketSendFD/RecvFD to return 0 on EAGAIN, or 1 on success * src/rpc/virnetclient.c: Move decoding of header & fds out of virNetClientCallDispatch and into virNetClientIOHandleInput. Handling blocking when sending/receiving FDs * src/rpc/virnetmessage.h: Add a 'donefds' field to track how many FDs we've sent / received * src/rpc/virnetserverclient.c: Handling blocking when sending/receiving FDs 2011-11-04 Eric Blake build: silence compiler warning on BSD Building on 64-bit FreeBSD 8.2 complained about a cast between a pointer and a smaller integer. Going through an intermediate cast shuts up the compiler. * src/util/threads-pthread.c (virThreadSelfID): Silence a warning. 2011-11-04 Eric Blake build: fix linking on BSD While building on FreeBSD (and after fixing a ptsname_r link error), I got this failure: ./.libs/libvirt_util.a(libvirt_util_la-threads.o)(.text+0x240): In function `virThreadCreate': util/threads-pthread.c:185: undefined reference to `pthread_create' It turns out that gnulib used only pthread_join for LIB_PTHREAD, but on FreeBSD, libc provides that (as a stub function); whereas the more complex pthread_create really does require -pthread, which gnulib tracked under [LT]LIBMULTITHREAD. * configure.ac (LIBS): Check LIBMULTITHREAD alongside LIB_PTHREAD. * src/Makefile.am (THREAD_LIBS): New variable. (libvirt_util_la_LIBADD, libvirt_lxc_LDADD): Use it. 2011-11-04 Laine Stump remote: fix mingw32 build tty is initialized, and later set in code that is compiled for all platforms, but is only used in a section that's inside #ifndef WIN32. 2011-11-04 Eric Blake lxc: avoid use-after-free I got this weird failure: error: Failed to start domain simple error: internal error cannot mix caller fds with blocking execution and tracked it down to a use-after-free - virCommandSetOutputFD was storing the address of a stack-local variable, which then went out of scope before the virCommandRun that dereferenced it. Bug introduced in commit 451cfd05 (0.9.2). * src/lxc/lxc_driver.c (lxcBuildControllerCmd): Move log fd registration... (lxcVmStart): ...to caller. 2011-11-04 Daniel P. Berrange Fix naming of constant for disk event All constants related to events should have a prefix of VIR_DOMAIN_EVENT_ * include/libvirt/libvirt.h.in, src/qemu/qemu_domain.c: Rename VIR_DOMAIN_DISK_CHANGE_MISSING_ON_START to VIR_DOMAIN_EVENT_DISK_CHANGE_MISSING_ON_START 2011-11-04 Jiri Denemark conf: Don't free uninitialized pointer This causes libvirtd to crash when both and are used in one domain XML. Introduced by 5fa3d775a9f8cdb3423373eb084219aaf778df11 2011-11-03 Eric Blake build: fix deep VPATH builds I ran into the following build failure: $ mkdir -p build1 build2/a/very/deep/hierarcy $ cd build2/a/very/deep/hierarcy $ ../../../../../configure && make $ cd ../../../../build1 $ ../configure && make ... ../../src/remote/remote_protocol.c:7:55: fatal error: ../../../../../src/remote/remote_protocol.h: No such file or directory Turns out that we were sometimes generating the remote_protocol.c file with information from the VPATH build, which is bad, since any file shipped in the tarball should be idempotent no matter how deep the VPATH build tree that created it. * src/rpc/genprotocol.pl: Don't embed VPATH into generated file. 2011-11-03 Philipp Hahn doc: Add capability. Allow /capabilities/guest/features/deviceboot. 2011-11-03 Eric Blake lxc: use common code for process cleanup Based on a Coverity report - the return value of waitpid() should always be checked, to avoid problems with leaking resources. * src/lxc/lxc_controller.c (lxcControllerRun): Use simpler virPidAbort. 2011-11-03 Daniel P. Berrange Fix default console type setting The default console type may vary based on the OS type. ie a Xen paravirt guests wants a 'xen' console, while a fullvirt guests wants a 'serial' console. A plain integer default console type in the capabilities does not suffice. Instead introduce a callback that is passed the OS type. * src/conf/capabilities.h: Use a callback for default console type * src/conf/domain_conf.c, src/conf/domain_conf.h: Use callback for default console type. Add missing LXC/OpenVZ console types. * src/esx/esx_driver.c, src/libxl/libxl_conf.c, src/lxc/lxc_conf.c, src/openvz/openvz_conf.c, src/phyp/phyp_driver.c, src/qemu/qemu_capabilities.c, src/uml/uml_conf.c, src/vbox/vbox_tmpl.c, src/vmware/vmware_conf.c, src/xen/xen_hypervisor.c, src/xenapi/xenapi_driver.c: Set default console type callback 2011-11-03 Daniel P. Berrange Set aliases for LXC/UML console devices To allow virDomainOpenConsole to access non-primary consoles, device aliases are required to be set. Until now only the QEMU driver has done this. Update LXC & UML to set aliases for any console devices * src/lxc/lxc_driver.c, src/uml/uml_driver.c: Set aliases for console devices 2011-11-03 Daniel P. Berrange Default console target type with no element When no element was set at all, the default console target type was not being honoured * src/conf/domain_conf.c: Set default target type for consoles with no 2011-11-03 Daniel P. Berrange Add support for multiple consoles in LXC Currently the LXC controller only supports setup of a single text console. This is wired up to the container init's stdio, as well as /dev/console and /dev/tty1. Extending support for multiple consoles, means wiring up additional PTYs to /dev/tty2, /dev/tty3, etc, etc. The LXC controller is passed multiple open file handles, one for each console requested. * src/lxc/lxc_container.c, src/lxc/lxc_container.h: Wire up all the /dev/ttyN links required to symlink to /dev/pts/NN * src/lxc/lxc_container.h: Open more container side /dev/pts/NN devices, and adapt event loop to handle I/O from all consoles * src/lxc/lxc_driver.c: Setup multiple host side PTYs 2011-11-03 Daniel P. Berrange Rewrite LXC I/O forwarding to use main event loop The current I/O code for LXC uses a hand crafted event loop to forward I/O between the container & host app, based on epoll to handle EOF on PTYs. This event loop is not easily extensible to add more consoles, or monitor other types of file descriptors. Remove the custom event loop and replace it with a normal libvirt event loop. When detecting EOF on a PTY, disable the event watch on that FD, and fork off a background thread that does a edge-triggered epoll() on the FD. When the FD finally shows new incoming data, the thread re-enables the watch on the FD and exits. When getting EOF from a read() on the PTY, the existing code would do waitpid(WNOHANG) to see if the container had exited. Unfortunately there is a race condition, because even though the process has closed its stdio handles, it might still exist. To deal with this the new event loop uses a SIG_CHILD handler to perform the waitpid only when the container is known to have actually exited. * src/lxc/lxc_controller.c: Rewrite the event loop to use the standard APIs. 2011-11-03 Daniel P. Berrange Fix crash formatting virtio console qemuBuildVirtioSerialPortDevStr was mistakenly accessing the target.name field in the virDomainChrDef object for chardevs belonging to a console. Those chardevs only have port set, and if there's > 1 console, the > 1port number results in trying to access a target.name with address 0x1 * src/qemu/qemu_command.c: Fix target.name handling and make code more robust wrt error reporting * src/qemu/qemu_command.c: Conditionally access target.name 2011-11-03 Daniel P. Berrange Allow multiple consoles per virtual guest While Xen only has a single paravirt console, UML, and QEMU both support multiple paravirt consoles. The LXC driver can also be trivially made to support multiple consoles. This patch extends the XML to allow multiple elements in the XML. It also makes the UML and QEMU drivers support this config. * src/conf/domain_conf.c, src/conf/domain_conf.h: Allow multiple devices * src/lxc/lxc_driver.c, src/xen/xen_driver.c, src/xenxs/xen_sxpr.c, src/xenxs/xen_xm.c: Update for internal API changes * src/security/security_selinux.c, src/security/virt-aa-helper.c: Only label consoles that aren't a copy of the serial device * src/qemu/qemu_command.c, src/qemu/qemu_driver.c, src/qemu/qemu_process.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Support multiple console devices * tests/qemuxml2xmltest.c, tests/qemuxml2argvtest.c: Extra tests for multiple virtio consoles. Set QEMU_CAPS_CHARDEV for all console /channel tests * tests/qemuxml2argvdata/qemuxml2argv-channel-virtio-auto.args, tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.args tests/qemuxml2argvdata/qemuxml2argv-console-virtio.args: Update for correct chardev syntax * tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.args, tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.xml: New test file 2011-11-03 Daniel P. Berrange Remove translations in socket test case The test case errors should not be translated since they're only targetted at developers, not users. * tests/virnetsockettest.c: Remove error reporting with translations 2011-11-03 Guido Günther virnetsockettest: Use a temporary directory in /tmp to avoid exceeding UNIX_PATH_MAX 2011-11-02 Guido Günther Use ENAMETOOLONG if the the socket path is longer than UNIX_PATH_MAX 2011-11-02 Eric Blake xen: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/xen/xen_hypervisor.c (xenHypervisorGetSchedulerParameters): Allow fewer than max. * src/xen/xend_internal.c (xenDaemonGetSchedulerParameters): Likewise. 2011-11-02 Eric Blake test: drop redundant check libvirt.c guarantees that nparams is non-zero for scheduler parameters. * src/test/test_driver.c (testDomainGetSchedulerParamsFlags): Drop redundant check. Avoid strcpy. 2011-11-02 Eric Blake lxc: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. Also, libvirt.c filters out nparams of 0 for scheduler parameters. * src/lxc/lxc_driver.c (lxcDomainGetMemoryParameters): Allow fewer than max. (lxcGetSchedulerParametersFlags): Drop redundant check. 2011-11-02 Eric Blake libxl: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/libxl/libxl_driver.c (libxlDomainGetSchedulerParametersFlags): Allow fewer than max. 2011-11-02 Eric Blake esx: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/esx/esx_driver.c (esxDomainGetMemoryParameters): Drop redundant check. (esxDomainGetSchedulerParametersFlags): Allow fewer than max. 2011-11-02 Eric Blake API: document scheduler parameter names Document the parameter names that will be used by virDomain{Get,Set}SchedulerParameters{,Flags}, rather than hard-coding those names in each driver, to match what is done with memory, blkio, and blockstats parameters. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SCHEDULER_CPU_SHARES) (VIR_DOMAIN_SCHEDULER_VCPU_PERIOD) (VIR_DOMAIN_SCHEDULER_VCPU_QUOTA, VIR_DOMAIN_SCHEDULER_WEIGHT) (VIR_DOMAIN_SCHEDULER_CAP, VIR_DOMAIN_SCHEDULER_RESERVATION) (VIR_DOMAIN_SCHEDULER_LIMIT, VIR_DOMAIN_SCHEDULER_SHARES): New field name macros. * src/qemu/qemu_driver.c (qemuSetSchedulerParametersFlags) (qemuGetSchedulerParametersFlags): Use new defines. * src/test/test_driver.c (testDomainGetSchedulerParamsFlags) (testDomainSetSchedulerParamsFlags): Likewise. * src/xen/xen_hypervisor.c (xenHypervisorGetSchedulerParameters) (xenHypervisorSetSchedulerParameters): Likewise. * src/xen/xend_internal.c (xenDaemonGetSchedulerParameters) (xenDaemonSetSchedulerParameters): Likewise. * src/lxc/lxc_driver.c (lxcSetSchedulerParametersFlags) (lxcGetSchedulerParametersFlags): Likewise. * src/esx/esx_driver.c (esxDomainGetSchedulerParametersFlags) (esxDomainSetSchedulerParametersFlags): Likewise. * src/libxl/libxl_driver.c (libxlDomainGetSchedulerParametersFlags) (libxlDomainSetSchedulerParametersFlags): Likewise. 2011-11-02 Daniel P. Berrange Fix off-by-one printing month in logging code The field 'mon' in 'struct tm' gives months 0-11, where as humans tend to expect months 1-12. Thus the month number needing adjusting by 1 * src/util/logging.c: Use human friendly month number 2011-11-02 Daniel P. Berrange Add missing param initialization in qemuDomainBlockStatsFlags * src/qemu/qemu_driver.c: Fix use of uninitialized 'params' 2011-11-02 Wen Congyang fix crash when starting network commit 27908453 introduces a regression, and it will cause libvirt crashed when starting network. The reason is that tapfd may be NULL, but we dereference it without checking whether it is NULL. 2011-11-01 Eric Blake qemu: allow getting < max typed parameters Since all virTypedParameter APIs allow us to return the number of slots we actually populated, we should allow the user to call with nparams too small (without overrunning their array) or too large (ignoring the tail of the array that we can't fill), rather than requiring that they get things exactly right. Making this change will make it easier for a future patch to introduce VIR_TYPED_PARAM_STRING, with filtering in libvirt.c rather than in every single driver, since users already have to be prepared for *nparams to be smaller on exit than on entry. * src/qemu/qemu_driver.c (qemuDomainGetBlkioParameters) (qemuDomainGetMemoryParameters): Allow variable nparams on entry. (qemuGetSchedulerParametersFlags): Drop redundant check. (qemudDomainBlockStats, qemudDomainBlockStatsFlags): Rename... (qemuDomainBlockStats, qemuDomainBlockStatsFlags): ...to this. Don't return unavailable stats. 2011-11-01 Eric Blake docs: improve typed parameter documentation virDomainBlockStatsFlags was missing a check that was present in virDomainGetMemoryParameters. Additionally, I found that the existing descriptions were a bit hard to read. A later patch will fix qemu to return fewer than max parameters if @nparams was too small on input. * src/libvirt.c (virDomainGetMemoryParameters) (virDomainGetBlkioParameters, virDomainGetSchedulerParameters) (virDomainGetSchedulerParametersFlags): Tweak documentation wording. (virDomainBlockStatsFlags): Likewise, and add sanity check. 2011-11-01 Daniel P. Berrange Don't overwrite error message during VM cleanup If an LXC VM fails to start, quite a few cleanup paths will result in the original error message being overwritten. Some other cleanup paths also forgot to actually terminate the VM. * src/lxc/lxc_driver.c: Ensure VM is terminated on startup failure and preserve original error 2011-11-01 Daniel P. Berrange Add support for probing filesystem with libblkid The LXC code for mounting container filesystems from block devices tries all filesystems in /etc/filesystems and possibly those in /proc/filesystems. The regular mount binary, however, first tries using libblkid to detect the format. Add support for doing the same in libvirt, since Fedora's /etc/filesystems is missing many formats, most notably ext4 which is the default filesystem Fedora uses! * src/Makefile.am: Link libvirt_lxc to libblkid * src/lxc/lxc_container.c: Probe filesystem format with libblkid 2011-11-01 Daniel P. Berrange Fix error message when failing to detect filesystem If we looped through /etc/filesystems trying to mount with each type and failed all options, we forget to actually raise an error message. * src/lxc/lxc_container.c: Raise error if unable to detect the filesystems. Also fix existing error message 2011-11-01 Daniel P. Berrange Workaround for broken kernel autofs mounts The kernel automounter is mostly broken wrt to containers. Most notably if you start a new filesystem namespace and then attempt to unmount any autofs filesystem, it will typically fail with a weird error message like Failed to unmount '/.oldroot/sys/kernel/security':Too many levels of symbolic links Attempting to detach the autofs mount using umount2(MNT_DETACH) will also fail with the same error. Therefore if we get any error on unmount()ing a filesystem from the old root FS when starting a container, we must immediately break out and detach the entire old root filesystem (ignoring any mounts below it). This has the effect of making the old root filesystem inaccessible to anything inside the container, but at the cost that the mounts live on in the kernel until the container exits. Given that SystemD uses autofs by default, we need LXC to be robust this scenario and thus this tradeoff is worthwhile. * src/lxc/lxc_container.c: Detach root filesystem if any umount operation fails. 2011-11-01 Daniel P. Berrange Correctly handle '*' in /etc/filesystems The /etc/filesystems file can contain a '*' on the last line to indicate that /proc/filessystems should be tried next. We have a check that this '*' only occurs on the last line. Unfortunately when we then start reading /proc/filesystems, we mistakenly think we've seen '*' in /proc/filesystems and fail * src/lxc/lxc_container.c: Skip '*' validation when we're reading /proc/filesystems 2011-11-01 Daniel P. Berrange Ensure errno is valid when returning from lxcContainerWaitForContinue Only some of the return paths of lxcContainerWaitForContinue will have set errno. In other paths we need to set it manually to avoid the caller getting a random stale errno value * src/lxc/lxc_container.c: Set errno in lxcContainerWaitForContinue 2011-11-01 Daniel P. Berrange Create /var/lib/libvirt/filesystems for LXC trees We already have a /var/lib/libvirt/images for OS install images. We need a separate /var/lib/libvirt/filesystems for OS install trees, since SELinux labelling will be different * libvirt.spec.in: Add /var/lib/libvirt/filesystems * src/Makefile.am: Create /var/lib/libvirt/filesystems 2011-11-01 Matthias Bolte esx: Support folders in the path of vpx:// connection URIs Allow the datacenter and compute resource parts of the path to be prefixed with folders. Therefore, the way the path is parsed has changed. Before, it was split in 2 or 3 items and the items' meanings were determined by their positions. Now the path can have 2 or more items and the the vCenter server is asked whether a folder, datacenter of compute resource with the specified name exists at the current hierarchy level. Before the datacenter and compute resource lookup automatically traversed folders during lookup. This is logic got removed and folders have to be specified explicitly. The proper datacenter path including folders is now used when accessing a datastore over HTTPS. This makes virsh dumpxml and define work for datacenters in folders. https://bugzilla.redhat.com/show_bug.cgi?id=732676 2011-11-01 Patrice LACHANCE esx: Support vSphere 5.x And virtual hardware version 8. 2011-11-01 Wen Ruo Lv Fix URI alias prefix matching with /etc/libvirt/libvirt.conf below: uri_aliases = [ "hail=qemu:///system", "sleet=qemu+ssh://root 9 115 122 57/system", "sam=qemu+unix:///system?socket=/var/run/libvirt/libvirt-sock", ] Neither "virsh -c hailly" nor "hai" should result in matching "hail=qemu:///system" Fix URI alias prefix matching when connecting 2011-11-01 Eric Blake docs: fix typo in / example * docs/formatdomain.html.in: Use dev, not def. Reported by Alexander Biryukov. 2011-11-01 Michal Privoznik ServerClient: Flush cached data If daemon is using SASL it reads client data into a cache. This cache is big (usually 65KB) and can thus contain 2 or more messages. However, on socket event we can dispatch only one message. So if we read two messages at once, the second will not be dispatched as the socket event goes away with filling the cache. Moreover, when dispatching the cache we need to remember to take care of client max requests limit. 2011-11-01 Daniel P. Berrange Fix storage pool source comparison to avoid comparing with self If we are comparing storage pools we must skip comparing with ourself, so that re-defining an existing pool works * conf/storage_conf.c: Skip self when comparing 2011-10-31 Sage Weil qemu: pass virConnectPtr into Domain{Attach,Detach}* The qemu RBD driver needs access to the conn in order to get the secret needed for connecting to the ceph cluster. 2011-10-31 Alex Jia qemu: plug memory leak Detected by Coverity. Leak introduced in commit 6cabc0b. * src/qemu/qemu_command.c: Clean up on failure. 2011-10-31 Matthias Bolte python: Fix documentation of virStream recv This was fixed in be757a3f7baf93b for libvirt.c. 2011-10-31 Michal Privoznik startupPolicty: Minor cleanups This patch does some cleanups to my previous startupPolicy patchset. 2011-10-30 Osier Yang qemu: Restore the original states of PCI device when restarting daemon To support "managed" mode of host PCI device, we record the original states (unbind_from_stub, remove_slot, and reprobe) so that could reattach the device to host with original driver. But there is no XML for theses attrs, and thus after daemon is restarted, we lose the original states. It's easy to reproduce: 1) virsh start domain 2) virsh attach-device dom hostpci.xml (in 'managed' mode) 3) service libvirtd restart 4) virsh destroy domain You will see the device won't be bound to the original driver if there was one. This patch is to solve the problem by introducing internal XML (won't be dumped to user, only dumped to status XML). The XML is: Which will be child node of .... (only for PCI device). A new struct "virDomainHostdevOrigStates" is introduced for the XML, and the according members are updated when preparing the PCI device. And function "qemuUpdateActivePciHostdevs" is modified to honor the original states. Use of qemuGetPciHostDeviceList is removed in function "qemuUpdateActivePciHostdevs", and the "managed" value of the device config is honored by the change. This fixes another problem alongside: qemuGetPciHostDeviceList set the device as "managed" force regardless of whether the device is configured as "managed='yes'" or not in XML, which is not right. 2011-10-29 Matthias Bolte vbox: Add support for VirtualBox 4.1 Deal with the incompatible changes in the VirtualBox 4.1 API. INetworkAdapter has its different AttachTo* method replaced by a settable attachmentType property. The maximum number of network adapters is now requestable per chipset type. The OpenMedium method got a bool parameter to request opening a medium under a new IID. 2011-10-29 Matthias Bolte vbox: Support shared folders Shared folders are handled as filesystems and can also be hotplugged. 2011-10-29 Matthias Bolte xenapi: Improve error reporting in xenapiOpen once again privP->session->error_description is a list and in order to get the complete error message all parts of the list should be concatenated. xenapiSessionErrorHandler does this when its third parameter is NULL. The current code discards all but the first part of the error message resulting in a potentially incomplete error message. This partly reverts 006be75ee214f9b4, that tried to avoid reporting a (null) in the error message. The actual problem is more general in returnErrorFromSession that might return NULL if there is no error. Make sure that returnErrorFromSession return non-NULL always. Also don't skip the last error message part. 2011-10-29 Roopa Prabhu macvtap: Fix error return value convention/inconsistencies - changed some return 1's to return -1 - changed if (rc) error checks to if (rc < 0) - fixed some other minor convention violations I might have missed some. Can fix in another patch or can respin Reported-by: Eric Blake Reported-by: Laine Stump 2011-10-29 Josh Durgin Use a common xml type for ceph secret usage. The types used in domaincommon.rng and secret.rng should be the same. Move genericName to basictypes.rng, then drop redundant types now that secret.rng uses basictypes.rng. 2011-10-28 Xu He Jie pci address conflict when virtio disk with drive type When using the xml as below: ------------------------------------------------------ /home/soulxu/data/work-code/qemu-kvm/x86_64-softmmu/qemu-system-x86_64