2018-03-01 Pavel Hrdina tests: introduce qemucapsfixreplies helper Sometimes we don't regenerate QEMU capabilities replies using QEMU binary but we simply add a new entry manually. In that case you need to manually fix all the replies ids. This helper will do that for you. Reviewed-by: Andrea Bolognani 2018-03-01 Michal Privoznik news: Update release notes Reviewed-by: Andrea Bolognani 2018-03-01 Andrea Bolognani src: Fix checking for clang The check was trying to use the shell variable $CC instead of the make variable $(CC); it also interpreted grep's return code wrong: 1 means the provided pattern was *not* matched. As a result, pdwtags was never run, not even when building with gcc. Reviewed-by: Daniel P. Berrangé 2018-03-01 John Ferlan conf: Fix crash in virDomainDefCompatibleDevice Commit id 'edae027c' blindly assumed that the passed @oldDev parameter would not be NULL when calling virDomainDeviceGetInfo; however, commit id 'b6a264e8' passed NULL for AttachDevice callers under the premise that there wouldn't be a device to check/update against. 2018-02-28 Daniel P. Berrangé travis: test upstart/systemd init script handling Enable testing of both the upstart and systemd init script handling. We test a different one in each scenario. Even though trusty only cares about upstart, it is fine for us to test rules that install systemd, since we're not actually running these scripts for real. Reviewed-by: Andrea Bolognani 2018-02-28 Daniel P. Berrangé travis: test "make install" and "make dist" on macOS We can't use "make distcheck" on macOS because many unit tests fail. We can still get coverage of some of the things "distcheck" validates, by running the "install" and "dist" targets. This is particularly useful because many conditional features are disabled on macOS, and this helps make sure we can still successfully install & dist when these bits are disabled. The default script is getting unreadable since it is all on one long line. Rather than adding further conditional clauses to it, we make use of the travis matrix config override for the script. Reviewed-by: Andrea Bolognani 2018-02-28 Daniel P. Berrangé travis: run "make distcheck" instead of just "make check" Running "make distcheck" includes the "make check", and "make dist" targets. It ensures that we have CLEANFILES and uninstall rules setup correctly, as well as validating VPATH builds succeed. Reviewed-by: Andrea Bolognani 2018-02-28 Daniel P. Berrangé travis: drop precise distro jobs The precise distro is marked deprecated in travis and will be dropped entirely in 2 months time. Reviewed-by: Andrea Bolognani 2018-02-28 Daniel P. Berrangé make: skip pdwtags when building with CLang When building with CLang the structs that are emitted by pdwtags appear in a completely different order than with GCC, which causes the comparison against expected data to fail. Ideally the test would not be sensitive to the ordering, because even future GCC could cause changes, but that's not easy to fix. So for now just skip the test when using clang. Reviewed-by: Andrea Bolognani 2018-02-28 Daniel P. Berrangé docs: remove legacy XHTML declaration We have switched the docs to using the HTML5 doctype declaration in commit b1c81567c7172bc9dcd701cf46ea3f87725d62c7 Author: Daniel P. Berrange Date: Wed Jul 26 18:01:25 2017 +0100 docs: switch to using HTML5 doctype declaration 2018-02-28 Michal Privoznik vshCommandOpt: Do more checking if skipChecks is set Currently if cmd->skipChecks is set (done only from completers) some basic checks are skipped because we're working over partially parsed command. See a26ff63ae4 for more detailed explanation. Anyway, the referenced commit was too aggressive in disabling checks and effectively returned success even in clear case of failure. For instance: # domif-getlink --interface causes virshDomainInterfaceCompleter() to be called, which calls virshDomainGetXML() which eventually calls vshCommandOptStringReq(.., name = "domain"); The --domain argument is required for the command and if not present -1 should be returned to tell the caller the argument was not found. Well, zero is returned meaning the argument was not found but it's not required either. 2018-02-28 Nikolay Shirokovskiy vz: fix ref/unref of domain obj in vzDomainRevertToSnapshot 2018-02-28 Julio Faracco virsh: fixing segfault by pool autocompleter function. The commands which requires a pool to perform any action for a volume is throwing a segfault when you pass the volume name before a pool name or without the argument '--pool'. An example that works: virsh # vol-list loops-pool Name Path ------------------------------------------------------------------- loop0 /mnt/loop0 virsh # vol-info --pool loops-pool lo An example that does not work: virsh # vol-list loops-pool Name Path ------------------------------------------------------------------- loop0 /mnt/loop0 virsh # vol-info lo Segmentation Fault The example 'vol-info' can be executed as 'vol-info loop0 --pool loops-pool'. So, this commit fixes this problem when the arguments are inverted and avoids the segfault. 2018-02-28 Zhangzijian qemu: fix memory leak of @vporttype during migration. 12 bytes in 1 blocks are definitely lost in loss record 188 of 1,145 at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) by 0x5D2CD77: xmlStrndup (in /lib/x86_64-linux-gnu/libxml2.so.2.7.8) by 0x514E137: virXMLPropString (virxml.c:506) by 0x234F51: qemuMigrationCookieNetworkXMLParse qemu_migration.c:1001) by 0x235FF8: qemuMigrationCookieXMLParse (qemu_migration.c:1333) by 0x236214: qemuMigrationCookieXMLParseStr (qemu_migration.c:1372) by 0x2365D2: qemuMigrationEatCookie (qemu_migration.c:1456) by 0x243DBA: qemuMigrationFinish (qemu_migration.c:6381) by 0x204032: qemuDomainMigrateFinish3 (qemu_driver.c:13228) by 0x521CCBB: virDomainMigrateFinish3 (libvirt-domain.c:4788) by 0x1936DE: remoteDispatchDomainMigrateFinish3 (remote.c:4580) by 0x16DBB1: remoteDispatchDomainMigrateFinish3Helper(remote_dispatch.h:7582) 2018-02-27 Laine Stump nwfilter: save error from DHCP snoop thread to report in main thread A problem encountered due to a bug in libpcap was reported to the caller as: An error occurred, but the cause is unknown This was because the error had been logged in the DHCPSnoop thread. The worker thread handling the API call to start a domain spins up the DHCPSnoop thread which watches for dhcp packets with libpcap, then uses virCondSignal() to notify the worker thread (which has been waiting with virCondWait()). The worker thread knows that there was an error (because threadStatus != THREAD_STATUS_OK), but the error info had been stored in thread-specific storage for the other thread, so the worker thread can only report that there was a failure, but it doesn't know why. The solution is to save the error that was logged (with virErrorPreserveLast() into the object the is used to share info between the threads, then we can set the error in the worker thread using virErrorRestore(). In the case of the error I was looking at, this changed the "unknown" message into: internal error: pcap_setfilter: can't remove kernel filter: Bad file descriptor 2018-02-27 Daniel P. Berrangé storage: fix sheepdog driver / test linking to avoid duplicating source The libvirt_storage_backend_sheepdog_priv.la library depends on symbols provided in the libvirt_driver_storage_impl.la library. As such the latter must be listed 2nd when passed to the linker to avoid symbol resolution problems. This mistake is being masked by the sheepdog driver linking in a second copy of the storage driver code. Remove this duplicate linkage of backend source and fix the test link order. make: fix uninstall of subdirectory data A typo in the uninstall-data-extra rule expansion meant we just called the install rule again, instead of the uninstall rule. While fixing this, just inline the dependancy, since the intermediate install-data-extra rule adds no value. po: refresh translations from zanata 2018-02-26 Nikolay Shirokovskiy lxc: fix rpc event leak on error path in virLXCControllerEventSend Reviewed-by: Daniel P. Berrangé daemon: fix rpc event leak on error path in remoteDispatchObjectEventSend Reviewed-by: Daniel P. Berrangé 2018-02-26 Nikolay Shirokovskiy vz: build fix: handle missing switch enum cases Ensure all enum cases are listed in switch statements, or cast away enum type in places where we don't wish to cover all cases. Build is broken after 67966ad51 [1]. [1] m4: enforce that all enum cases are listed in switch statements Reviewed-by: Daniel P. Berrangé 2018-02-26 Nikolay Shirokovskiy vz: build fix after excluding cross driver headers Build is broken by 5529b057 [1]. [1] cfg: forbid includes of headers in network and storage drivers again Reviewed-by: Daniel P. Berrangé 2018-02-26 Michal Privoznik qemu: Add virConnectPtr back to some migration methods This partially reverts 82592551cb8c4112cfa2264d50b8dce5349533d5. When migrating a domain, qemuMigrationDstPrepareAny() is called which eventually calls qemuProcessLaunch(conn = NULL, flags = VIR_QEMU_PROCESS_START_AUTODESTROY); But the very first thing that qemuProcessLaunch does is check if AUTODESTROY flag is set and @conn is not NULL. Well, it is. Reviewed-by: Daniel P. Berrangé 2018-02-26 Michal Privoznik qemuProcessLaunch: Print all arguments to debug Reviewed-by: Daniel P. Berrangé 2018-02-26 Michal Privoznik qemuMigrationSrcIsSafe: Check local storage more thoroughly https://bugzilla.redhat.com/show_bug.cgi?id=1494454 If a domain disk is stored on local filesystem (e.g. ext4) but is not being migrated it is very likely that domain is not able to run on destination. Regardless of share/cache mode. Reviewed-by: Daniel P. Berrangé 2018-02-23 Jim Fehlig libxl: round memory values to next 1MiB increment libxl requires the memory sizes to be rounded to 1MiB increments. Attempting to start a domain that violates this requirement will fail with the marginally helpful error 2018-02-22 01:55:32.921+0000: xc: panic: xc_dom_boot.c:141: xc_dom_boot_mem_init: can't allocate low memory for domain: Out of memory 2018-02-22 01:55:32.921+0000: libxl: libxl_dom.c:671:libxl__build_dom: xc_dom_boot_mem_init failed: No such file or directory Round the maximum and current memory values to the next 1MiB increment when generating the libxl_domain_config object. 2018-02-23 Daniel P. Berrangé make: split lxc driver build rules into lxc/Makefile.inc.am make: split vz driver build rules into vz/Makefile.inc.am make: split xenapi driver build rules into xenapi/Makefile.inc.am make: split xen driver build rules into xen/Makefile.inc.am make: split libxl driver build rules into libxl/Makefile.inc.am make: split xenconfig driver build rules into xenconfig/Makefile.inc.am make: split bhyve driver build rules into bhyve/Makefile.inc.am make: split qemu driver build rules into qemu/Makefile.inc.am make: split openvz driver build rules into openvz/Makefile.inc.am make: split vbox driver build rules into vbox/Makefile.inc.am make: split vmware driver build rules into vmware/Makefile.inc.am make: split hyperv driver build rules into hyperv/Makefile.inc.am make: split ESX driver build rules into esx/Makefile.inc.am make: split test driver build rules into test/Makefile.inc.am make: split PHyp driver build rules into phyp/Makefile.inc.am make: split UML driver build rules into uml/Makefile.inc.am 2018-02-23 Christian Ehrhardt tools: avoid text spilling into variables While libvirt-guests.sh is running cases can let guest_is_on fail which causes check_guests_shutdown to print output. That output shall not spill into the users of function check_guests_shutdown which is therefore now returning values in a variable like guest_is_on already did. Original-Author: Christian Ehrhardt Modified-By: Jorge Niedbalski 2018-02-23 Daniel P. Berrangé make: fix VPATH install of upstart files 2018-02-23 Jiri Denemark qemu: Fix updating device with boot order Commit v3.7.0-14-gc57f3fd2f8 prevented adding a element to an inactive domain with global element. However, as a result of that change updating any device with boot order would fail with 'boot order X is already used by another device', where "another device" is in fact the device which is being updated. To fix this we have to ignore the device which we're about to update when checking for boot order conflicts. https://bugzilla.redhat.com/show_bug.cgi?id=1546971 2018-02-23 Jiri Denemark Pass oldDev to virDomainDefCompatibleDevice on device update When calling virDomainDefCompatibleDevice to check a new device during device update, we need to pass the original device which is going to be updated in addition to the new device. Otherwise, the function can report false conflicts. The new argument is currently ignored by virDomainDefCompatibleDevice, but this will change in the following patch. https://bugzilla.redhat.com/show_bug.cgi?id=1546971 2018-02-23 Jiri Denemark lxc: Drop useless check in live device update Checking the new device definition makes little sense when lxc driver does not support live device update at all. qemu_hotplug: Drop dead code in net update vm->def->nets[changeidx] can never be NULL for changeidx returned by virDomainNetFindIdx. 2018-02-23 Michal Privoznik libvirt.spec.in: Don't distribute libvirtd.upstart Firstly, for rpm we are building libvirt with --init-script=systemd or --init-script=redhat. So upstart is never enabled. And only due to a bug we installed libvirtd.upstart file. Reviewed-by: Pavel Hrdina 2018-02-22 Jim Fehlig news: libxl now supports setting clock offset and adjustment 2018-02-22 Daniel P. Berrangé make: fix long line in makefile that violate syntax-check rules make: fix another VPATH bug impacting install of sysconf files 2018-02-22 Michal Privoznik src: Clean *.logrotate, .aug and libvirtd.policy files on 'make clean' Before 3f055b5997c we were doing that. However after the commit we are leaving the files behind. 2018-02-22 Daniel P. Berrangé make: fix VPATH build for libvirtd augeas check 2018-02-22 John Ferlan conf,qemu: Check for NULL addrs in virDomainUSBAddressEnsure Rather than having the caller check, if the input @addrs is NULL (e.g. priv->usbaddrs), then just return 0. This also removes the need for ATTRIBUTE_NONNULL which only really helped if someone passed a NULL as a parameter not if the passed parameter is NULL. conf,qemu: Check for NULL addrs in virDomainUSBAddressRelease Rather than having the caller check, if the input @addrs is NULL (e.g. priv->usbaddrs), then just return 0. This also removes the need for ATTRIBUTE_NONNULL which only really helped if someone passed a NULL as a parameter not if the passed parameter is NULL. 2018-02-22 Nikolay Shirokovskiy port allocator: make port range constant object port allocator: remove release functionality from set used Let's use virPortAllocatorRelease instead of virPortAllocatorSetUsed(false). port allocator: drop skip bind check flag This flag is only used for tests. Let's instead overload bind syscall in mocks where it is not done yet. port allocator: remove range check in release function Range check in virPortAllocatorSetUsed is not useful anymore when we manage ports for entire unsigned short range values. port allocator: remove range on manual port reserving Range check in virPortAllocatorSetUsed is not useful anymore when we manage ports for entire unsigned short range values. 2018-02-22 Nikolay Shirokovskiy port allocator: make used port bitmap global Host tcp4/tcp6 ports is a global resource thus we need to make port accounting also global or we have issues described in [1] when port allocator ranges of different instances are overlapped (which is by default for qemu for example). Let's have only one global port allocator object that take care of the entire ports range (0 - 65535) and introduce port range object for clients to specify desired auto allocation band. [1] https://www.redhat.com/archives/libvir-list/2017-December/msg00600.html 2018-02-22 Daniel P. Berrangé rpm: don't enable fuse on RHEL-6 Fuse was recently enabled whereever LXC is enabled: commit 34783a9e6b9c3d8850db878fc323c1ff40703ea6 Author: Jiri Denemark Date: Fri Feb 9 13:42:50 2018 +0100 spec: Enable fuse only if LXC is enabled Unfortunately the version of Fuse in RHEL-6 is too old for libvirt's needs, but we still have LXC enabled there. 2018-02-22 Daniel P. Berrangé daemon: move logrotate files to src/remote/ daemon: move misc libvirtd policy files to src/remote/ daemon: move configuration files to src/remote daemon: move init system files into src/remote/ Move the systemd, sysvinit, upstart and sysconfig files into the src/remote/ directory. daemon: move libvirtd POD manpage into src/remote/ daemon: move libvirtd code into src/remote/ directory Having a daemon/ directory makes little sense from a code structure point of view, as 90% of the code that is built into libvirtd already lives in the src/ directory. The virtlockd and virlogd daemons also live entirely in src/{locking,logging} directories. This moves the source code for libvirtd into src/remote/, alongside the client code. daemon: remove obsolete doc describing daemon threading The THREADS.txt describes the way threading worked in libvirtd many many years ago, long before even the RPC code was modularized. Things have evolved significantly since then, so delete this potentially misleading doc. src: simplify sysconfig file handling in make rules Remove lots of duplication in the sysconfig file handling, so we can add more conf files without modifying so many places. src: simplify sysv init file handling in make rules Remove lots of duplication in the sysv init file handling, so we can add more init files without modifying so many places. src: simplify systemd unit file handling in make rules Remove lots of duplication in the systemd unit file handling, so we can add more unit files without modifying so many places. bhyve: add missing cases for graphics listen type Address warning from -Wswitch-enum by adding missing cases for graphics listen types that are not supported. 2018-02-21 Jim Fehlig libxl: add support for specifying clock offset and adjustment libxl supports setting the domain real time clock to local time or UTC via the localtime field of libxl_domain_build_info. Adjustment of the clock is also supported via the rtc_timeoffset field. The libvirt libxl driver has never supported these settings, instead relying on libxl's default of a UTC real time clock with adjustment set to 0. There is at least one user that would like the ability to change the defaults https://www.redhat.com/archives/libvirt-users/2018-February/msg00059.html Add support for specifying a local time clock and for specifying an adjustment for both local time and UTC clocks. Add a test case to verify the XML to libxl_domain_config conversion. Local time clock and clock adjustment is already supported by the XML <-> xl.cfg converter. What is missing is an explicit test for the conversion. There are plenty of existing tests that all use UTC with 0 adjustment. Hijack test-fullvirt-tsc-timer to test a local time clock with 1 hour adjustment. Reviewed-by: Daniel P. Berrangé 2018-02-21 Ján Tomko virLogGetOutputs: remove unnecessary braces Commit 9275def reduced the if block to one line without removing the braces. 2018-02-21 Daniel P. Berrangé m4: enforce that all enum cases are listed in switch statements As a general rule any time we switch() on something that is an enum, we want to have a case for every enum constant. The -Wswitch warning will report any switch where we've violated this rule, except if that switch has a default case. Unfortunately it is reasonable to want to list all enum constants *and* also have a default case. To get a warning in that scenario requires that we turn on -Wswitch-enum. In a few cases where we explicitly don't want to list all enum cases, we can discard the enum type checking by casting the value to a plain int. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé tools: handle missing switch enum cases Cast away enum type in places where we don't wish to cover all cases. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé xen: handle missing switch enum cases Ensure all enum cases are listed in switch statements. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé security: handle missing switch enum cases Ensure all enum cases are listed in switch statements. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé rpc: handle missing switch enum cases Ensure all enum cases are listed in switch statements. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé qemu: handle missing switch enum cases Ensure all enum cases are listed in switch statements, or cast away enum type in places where we don't wish to cover all cases. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé nwfilter: handle missing switch enum cases Ensure all enum cases are listed in switch statements, or cast away enum type in places where we don't wish to cover all cases. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé lxc: handle missing switch enum cases Ensure all enum cases are listed in switch statements, or cast away enum type in places where we don't wish to cover all cases. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé libxl: handle missing switch enum cases Cast away enum type for libxl scheduler constants since we don't want to cover all of them and don't want build to break when new ones are added. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé hyperv: handle missing switch enum cases Ensure all enum cases are listed in switch statements. This improves debug logging integration with openwsman. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé esx: handle missing switch enum cases Ensure all enum cases are listed in switch statements, or explicitly cast away enum type where we don't want to list all cases. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé conf: handle missing switch enum cases Ensure all enum cases are listed in switch statements. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé util: handle missing switch enum cases Ensure all enum cases are listed in switch statements. Reviewed-by: John Ferlan 2018-02-21 Daniel P. Berrangé util: add a virReportEnumRangeError for bad value reporting To ensure we have standardized error messages when reporting problems with enum values being out of a range, add virReportEnumRangeError(). virReportEnumRangeError(virDomainState, 34); results in a message "internal error: Unexpected enum value 34 for virDomainState" 2018-02-21 Daniel P. Berrangé src: remove WITH_LIBVIRTD condition around hal/udev build Currently building --without-libvirtd causes a failure to link the node device driver: node_device/.libs/libvirt_driver_nodedev_la-node_device_driver.o: In function `nodedevRegister': /home/berrange/src/virt/libvirt/src/node_device/node_device_driver.c:649: undefined reference to `udevNodeRegister' collect2: error: ld returned 1 exit status because it causes us to build the core nodedev driver, but then skip the implementations, despite udev being available. 2018-02-20 Laine Stump nwfilter: assure virNWFilterSnoop(Eth|Dhcp)Hdr structs don't change size These two objects are used to access fields in actual ethernet packets captures with libpcap, so it's essential that they don't change size for any reason. This patch uses gnulib's verify() macro to make sure their sizes don't change. Reviewed-by: Daniel P. Berrangé 2018-02-20 Daniel P. Berrangé Fix build with GCC 8 new switch fallthrough warnings GCC 8 became more fussy about detecting switch fallthroughs. First it doesn't like it if you have a fallthrough attribute that is not before a case statement. e.g. FOO: BAR: WIZZ: ATTRIBUTE_FALLTHROUGH; Is unacceptable as there's no final case statement, so while FOO & BAR are falling through, WIZZ is not falling through. IOW, GCC wants us to write FOO: BAR: ATTRIBUTE_FALLTHROUGH; WIZZ: Second, it will report risk of fallthrough even if you have a case statement for every single enum value, but only if the switch is nested inside another switch and the outer case statement has no final break. This is is arguably valid because despite the fact that we have cast from "int" to the enum typedef, nothing guarantees that the variable we're switching on only contains values that have corresponding switch labels. e.g. int domstate = 87539319; switch ((virDomainState)domstate) { ... } will not match enum value, but also not raise any kind of compiler warning. So it is right to complain about risk of fallthrough if no default: is present. 2018-02-20 Daniel P. Berrangé conf: add enum constants for default controller models The controller model is slightly unusual in that the default value is -1, not 0. As a result the default value is not covered by any of the existing enum cases. This in turn means that any switch() statements that think they have covered all cases, will in fact not match the default value at all. In the qemuDomainDeviceCalculatePCIConnectFlags() method this has caused a serious mistake where we fallthrough from the SCSI controller case, to the VirtioSerial controller case, and from the USB controller case to the IDE controller case. By adding explicit enum constant starting at -1, we can ensure switches remember to handle the default case. Reviewed-by: John Ferlan 2018-02-20 Andrea Bolognani qemu: Simplify modelName stringification There's no need to perform checks before conversion, we can just call virDomainControllerPCIModelNameTypeToString() and check the results later on. Since the variables involved are only used for PCI controllers, we can declare them in the 'case' scope rather than in the function scope to make everything a bit nicer while at it. 2018-02-20 Andrea Bolognani qemu: Move skip for implicit PHB of pSeries guests Performing the skip earlier will help us making the function nicer later on. We also make the condition for the skip a bit more precise, though that'a more for self-documenting purposes and doesn't change anything in practice. qemu: Move 'done' label in qemuBuildControllerDevStr() Even when we skip part of the processing, we still want error checking on the buffer. 2018-02-20 Michal Privoznik storage_conf: Make virStorageAuthDefFormat return void This function returns nothing but zero. Therefore it makes no sense to have it returning an integer. Reviewed-by: Daniel P. Berrangé 2018-02-20 Michal Privoznik virDomainDiskSourceFormatInternal: Avoid leaking @childBuf If formatting of storage encryption or private data fails we must jump to the error label instead of returning immediately otherwise @attrBuf and @childBuf might be leaked. Reviewed-by: Daniel P. Berrangé 2018-02-20 Daniel P. Berrangé daemon: trigger RPC re-generation when Makefile.am changes The src/Makefile.am rules all re-generate the RPC dispatch code whenever the Makefile.am changes, so for consistency do that for daemon/Makefile.am too. Reviewed-by: Pavel Hrdina 2018-02-20 Andrea Bolognani docs: Document pcie-root requirement for q35 guests When you add a bunch of pcie-root-port controllers to a q35 guest in order to have hotplug capabilities, you also need to make sure you're adding the pcie-root controller at the same time or you will get an error. Document this fact. Reviewed-by: Daniel P. Berrangé 2018-02-20 Laine Stump tests: fix bhyve build This file was modified in an editor buffer but not saved prior to commit e62cb4a9b78 (which removed virMacAddr::generated), so the bhyve build would fail. 2018-02-19 Laine Stump conf: move 'generated' member from virMacAddr to virDomainNetDef Commit 7e62c4cd26d (first appearing in libvirt-3.9.0 as a resolution to rhbz #1343919) added a "generated" attribute to virMacAddr that was set whenever a mac address was auto-generated by libvirt. This knowledge was used in a single place - when trying to match a NetDef from the Domain to Delete with user-provided XML. Since the XML parser always auto-generates a MAC address for NetDefs when none is provided, it was previously impossible to make a search where the MAC address isn't significant, but the addition of the "generated" attribute made it possible for the search function to ignore auto-generated MACs. This implementation had a problem though - it was adding a field to a "low level" struct - virMacAddr - which is used in other places with the assumption that it contains exactly a 6 byte MAC address and nothing else. In particular, virNWFilterSnoopEthHdr uses virMacAddr as part of the definition of an ethernet packet header, whose layout must of course match an actual ethernet packet. Adding the extra bools into virNWFilterSnoopEthHdr caused the nwfilter driver's "IP discovery via DHCP packet snooping" functionality to mysteriously stop working. In order to fix that behavior, and prevent potential future similar odd behavior, this patch moves the "generated" member out of virMacAddr (so that it is again really is just a MAC address) into virDomainNetDef, and sets it only when virDomainNetGenerateMAC() is called from virDomainNetDefParseXML() (which is the only time we care about it). Resolves: https://bugzilla.redhat.com/1529338 (It should also be applied to any maintenance branch that applies commit 7e62c4cd26 and friends to resolve https://bugzilla.redhat.com/1343919) 2018-02-19 Andrea Bolognani docs: Fix indentation of inlined JavaScript snippet 2018-02-19 Daniel P. Berrangé qemu: rename migration APIs to include Src or Dst in their name It is very difficult while reading the migration code trying to understand whether a particular function is being called on the src side or the dst side, or either. Putting "Src" or "Dst" in the method names will make this much more obvious. "Any" is used in a few helpers which can be called from both sides. Reviewed-by: John Ferlan 2018-02-19 Peter Krempa qemu: blockcopy: Add check for bandwidth QEMU code does not work well with too big numbers on the JSON monitor so our monitor code supports sending only numbers up to LLONG_MAX. Avoid a weird error message by limiting the size of the 'bandwidth' parameter for block copy. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1532542 2018-02-19 Peter Krempa virsh: Fix internal naming of some blockjob commands The variable names for the options and information about a command should have an underscore in places where the virsh command has a hyphen. The function callback name should capitalize the letter after the hyphen. This was not used in 'blockcommit', 'blockcopy', 'blockjob', 'blockpull', and 'blockresize' commands. 2018-02-19 Andrea Bolognani bhyve: Fix build Commit 2d43f0a2dcfd dropped virDomainDiskTranslateSourcePool()'s first argument but failed to update callers in the bhyve driver. 2018-02-19 Daniel P. Berrangé tests: remove networkRegister() call from qemuxml2argvtest Further cleanup from commit 0c63c117a2d17f66b05dd83e50aa36ac0b0c9843 Author: Daniel P. Berrangé Date: Fri Feb 9 15:08:53 2018 +0000 conf: reimplement virDomainNetResolveActualType in terms of public API 2018-02-19 Daniel P. Berrangé tests: drop linkage to libvirt_driver_network_impl.la The qemuxml2argvtest does not need to link to the network driver after this commit: commit 0c63c117a2d17f66b05dd83e50aa36ac0b0c9843 Author: Daniel P. Berrangé Date: Fri Feb 9 15:08:53 2018 +0000 conf: reimplement virDomainNetResolveActualType in terms of public API 2018-02-19 Daniel P. Berrangé qemu: don't export migration job APIs These APIs are not required anywhere outside the migration code so need not be exported to the rest of the QEMU driver. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé qemu: remove virConnectPtr in some migration methods The qemuMigrationPrecreateStorage method needs a connection to access the storage driver. Instead of passing it around, open it at time of use. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé qemu: remove virConnectPtr from some more startup code paths There's a few places in startup code paths which pass around a virConnectPtr which is no longer required. Specifically, the qemuProcessStart() method now only requires a non-NULL connection if autodestroy is requested. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé qemu: stop passing in virConnectPtr for looking up networks When setting up graphics, we sometimes need to resolve networks, requiring the caller to pass in a virConnectPtr, except sometimes they pass in NULL. Use virGetConnectNetwork() to acquire the connection to the network driver when it is needed. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé qemu: don't pass virConnectPtr around for secrets During domain startup there are many places where we need to acquire secrets. Currently code passes around a virConnectPtr, except in the places where we pass in NULL. So there are a few codepaths where ability to start guests using secrets will fail. Change to acquire a handle to the secret driver when needed. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé conf: stop passing virConnectPtr into virDomainDiskTranslateSourcePool Rather than expecting callers to pass a virConnectPtr into the virDomainDiskTranslateSourcePool() method, just acquire a connection to the storage driver when needed. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé qemu: stop passing virConnectPtr into qemuMonitorStartCPUs There is a long standing hack to pass a virConnectPtr into the qemuMonitorStartCPUs method, so that when the text monitor prompts for a disk password, we can lookup virSecretPtr objects. This causes us to have to pass a virConnectPtr around through countless methods up the call chain....except some places don't have any virConnectPtr available so have always just passed NULL. We can finally fix this disastrous design by using virGetConnectSecret() to open a connection to the secret driver at time of use. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé conf: reimplement virDomainNetResolveActualType in terms of public API Now that we have the ability to easily open connections to secondary drivers, eg network:///system, it is possible to reimplement the virDomainNetResolveActualType method in terms of the public API. This avoids the need to have the network driver provide a callback for it. Reviewed-by: John Ferlan 2018-02-19 Daniel P. Berrangé driver: allow override of connection for secondary drivers When the test suite is running, we don't want to be triggering the startup of daemons for the secondary drivers. Thus we must provide a way to set a custom connection for the secondary drivers, to override the default logic which opens a new connection. This will also be useful for code where we have a whole set of separate functions calls all needing the secret driver. Currently the connection to the secret driver is opened & closed many times in quick succession. This will allow us to pre-open a connection temporarily, improving the performance of startup. Reviewed-by: John Ferlan 2018-02-17 Zhuang Yanying news: Add support for setting Chassis SMBIOS data fields Reviewed-by: John Ferlan 2018-02-17 Zhuang Yanying qemu: Generate SMBIOS Chassis strings command line This wires up the previously added Chassis strings XML schema to be able to generate comamnd line args for QEMU. This requires QEMU >= 2.1 release containing this patch: SMBIOS: Build aggregate smbios tables and entry point https://git.qemu.org/?p=qemu.git;a=commit;h=c97294ec1b9e36887e119589d456557d72ab37b5 Reviewed-by: John Ferlan 2018-02-17 Zhuang Yanying conf: Add support for setting Chassis SMBIOS data fields This type of information defines attributes of a system chassis, such as SMBIOS Chassis Asset Tag. access inside VM (for example) Linux: /sys/class/dmi/id/chassis_asset_tag. Windows: (Get-WmiObject Win32_SystemEnclosure).SMBIOSAssetTag wirhin Windows PowerShell. As an example, add the following to the guest XML Dell Inc. 2.12 65X0XF2 40000101 Type3Sku1 Reviewed-by: John Ferlan 2018-02-16 Daniel P. Berrangé m4: disable gcc8 -Wcast-function-type warnings from -Wextra The -Wextra flag bundle gained a new warning -Wcast-function-type. This complains if you cast between two function prototypes where the number of parameters or their data types are not compatible. Unfortunately we need such "bad" function casts for our event callbacks. It is possible to silence the warning by first casting to the generic "void (*)(void)" function prototype, but that is rather ugly to add throughout libvirt code. 2018-02-14 Peter Krempa virsh: Remove sub-element in virshFindDisk Previously we've removed the data only in virshUpdateDiskXML when changing the disk source for the CDROM since the backing store would be invalid. Move the code into a separate function and callit from virshFindDisk which is also used when detaching disk. The detaching code does not necessarily need to get the full backing chain since it will need to act on the one managed by libvirt anyways and this also takes care of problems when parts of the backing store were invalid due to buggy RBD detection code. 2018-02-14 Peter Krempa util: storage: Remove detected authentication data for backing chains We can't really detect all the authentication data in a sane manner for disk backing chains. Since the old RBD parser parses it in some cases as the argv->XML convertor requires it, we can't just drop it. Instead clear any detected authentication data in the code paths related to disk backing chain lookup and fix the tests to cope with the change. https://bugzilla.redhat.com/show_bug.cgi?id=1544659 2018-02-14 Peter Krempa virsh: detach-disk: Add --print-xml switch Similarly to other commands add an argument which allows to check the XML which would be used to execute the operation instead. virstoragetest: Add test case for NBD over unix socket with new syntax Use the new syntax which uses the 'UnixSocket' type in qemu. 2018-02-14 Peter Krempa storage: Fix formatting and parsing of qemu type 'UnixSocketAddress' The documentation for the JSON/qapi type 'UnixSocketAddress' states that the unix socket path field is named 'path'. Unfortunately qemu uses 'socket' in case of the gluster driver (despite documented otherwise). Add logic which will format the correct fields while keeping support of the old spelling. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1544325 2018-02-14 Jiri Denemark spec: Drop overlapping triggers The postun trigger for libvirt-daemon was defined twice for overlapping ranges of package verions if systemd support was switched off (which happens when building on something ancient, such as RHEL-6). Let's combine the two triggers into the one which is called when libvirt-daemon < 1.3.0 is uninstalled. As a side effect, virtlockd and virtlogd might be reloaded twice after an upgrade from libvirt newer than 1.2.1 and older than 1.3.0 (by postun script from the old libvirt and postun trigger from the new libvirt). Reviewed-by: Daniel P. Berrangé 2018-02-14 Jiri Denemark spec: Fix indentation in daemon's triggerpostun Reviewed-by: Daniel P. Berrangé spec: Prepare for future RHEL Reviewed-by: Daniel P. Berrangé 2018-02-14 Jiri Denemark spec: Drop checks for old Fedora releases The oldest Fedora release supported by the spec file is 26. Checking for anything older makes no sense. Reviewed-by: Daniel P. Berrangé 2018-02-14 Jiri Denemark spec: Build virt-login-shell iff LXC driver is enabled Building virt-login-shell doesn't really make any sense without LXC and doing so even breaks "make rpm" since the associated files are installed but unpackaged (the login-shell sub package already depends on LXC). Reviewed-by: Daniel P. Berrangé 2018-02-14 Jiri Denemark spec: Enable fuse only if LXC is enabled Enabling fuse without LXC does not make a lot of sense because fuse is used only by LXC. Reviewed-by: Daniel P. Berrangé 2018-02-14 Michal Privoznik qemu: Check for down limit of SLIRP prefix too https://bugzilla.redhat.com/show_bug.cgi?id=1515533 We're already checking if IPv4 prefix isn't too long. But we are not checking if it isn't too short. QEMU supports prefixes longer than 4 (including). I haven't find anything similar related to IPv6 in qemu sources. Reviewed-by: John Ferlan 2018-02-14 Peter Krempa qemu: block: Remove misleading part of comment in qemuBlockStorageSourceBuildJSONSocketAddress The array indexes are formatted if the JSON->commandline translator is translating an array type. It does not at all depend on this function. 2018-02-13 Bjoern Walk tests: run virshtest independent of current pwd virshtest execves the virsh binary. Make sure that it finds the binary's location independent of the current working directory by specifying the absolute path as determined by the build environment. Reviewed-by: Marc Hartmayer Reviewed-by: Boris Fiuczynski 2018-02-13 Andrea Bolognani tests: Clean up HPT tests Give them better names and remove some redundancy. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani conf: Improve HPT feature handling Instead of storing separately whether the feature is enabled or not and what resizing policy should be used, store both of them in a single place. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani conf: Improve IOAPIC feature handling Instead of storing separately whether the feature is enabled or not and what driver should be used, store both of them in a single place. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani qemu: Fix GIC behavior for the default case When no GIC version is specified, we currently default to GIC v2; however, that's not a great default, since guests will fail to start if the hardware only supports GIC v3. Change the behavior so that a sensible default is chosen instead. That basically means using the same algorithm whether the user didn't explicitly enable the GIC feature or they explicitly enabled it but didn't specify any GIC version. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani tests: Improve GIC tests Account for the fact that the default might change based on what GIC versions are supported by QEMU. That's not the case at the moment, but it will be soon. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani conf: Integrate all features ABI checks in the switch There are a few stray checks which still live outside of the switch in virDomainDefFeaturesCheckABIStability() for no good reason. Move them inside the switch, and update the error messages to be consistent while at it. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani conf: Validate VIR_DOMAIN_FEATURE_CAPABILITIES properly Unlike most other features, VIR_DOMAIN_FEATURE_CAPABILITIES is of type virDomainCapabilitiesPolicy instead of virTristateSwitch, so we need to handle it separately for the error message to make sense. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani conf: Use switch in virDomainDefFeaturesCheckABIStability() The compiler can make sure we are handling all features. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani qemu: Move GIC checks to qemuDomainDefValidateFeatures() Keep them along with other arch/machine type checks for features instead of waiting until command line generation time. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani qemu: Use switch in qemuDomainDefValidateFeatures() The compiler can make sure we are handling all features. While reworking the logic, also change error messages to a more consistent style. Reviewed-by: John Ferlan 2018-02-13 Andrea Bolognani qemu: Move feature verification from PostParse() to Validate() We want to perform all feature verification in a single spot, but some of it (eg. GIC) is currently being performed at command line generation time, and moving it to PostParse() would cause guests to disappear. Moving verification to Validate() allows us to side-step the issue. Reviewed-by: John Ferlan 2018-02-12 Andrea Bolognani qemu: Fix indentation in qemuBuildControllerDevStr() Add braces around the multi-line body as well, in compliance with our coding style. 2018-02-12 Daniel P. Berrangé Revert "qemu: Expose rx/tx_queue_size in qemu.conf too" This reverts commit 038eb472a0d970a17ccf4343ead0666df5c92f9d. On reflection adding defaults for arbitrary guest XML device config settings to the qemu.conf is not a sustainable path. Removing the support for rx/tx queue size so that it doesn't set a bad precedent. 2018-02-12 John Ferlan qemu: Alter condition to avoid possible NULL deref Commit 'f0f2a5ec2' neglected to adjust the if condition to split out the possibility that the @watchdog is NULL when altering the message to add detail about the model. Just split out the condition and use previous/original message, but with the new message code. Found by Coverity 2018-02-12 Daniel P. Berrangé tests: fix running of qemuxml2argvtest program The previous commit: commit a455d41e3e1c1af3a36ccdbb2e3f2356cc58993e Author: Daniel P. Berrangé Date: Thu Jan 25 09:35:50 2018 +0000 conf: expand network device callbacks to cover resolving NIC type mistakenly dropped qemuxml2argvtest from the tests due to a typo. 2018-02-12 Daniel P. Berrangé log: fix deadlock obtaining hostname (related CVE-2018-6764) The fix for CVE-2018-6764 introduced a potential deadlock scenario that gets triggered by the NSS module when virGetHostname() calls getaddrinfo to resolve the hostname: #0 0x00007f6e714b57e7 in futex_wait #1 futex_wait_simple #2 __pthread_once_slow #3 0x00007f6e71d16e7d in virOnce #4 0x00007f6e71d0997c in virLogInitialize #5 0x00007f6e71d0a09a in virLogVMessage #6 0x00007f6e71d09ffd in virLogMessage #7 0x00007f6e71d0db22 in virObjectNew #8 0x00007f6e71d0dbf1 in virObjectLockableNew #9 0x00007f6e71d0d3e5 in virMacMapNew #10 0x00007f6e71cdc50a in findLease #11 0x00007f6e71cdcc56 in _nss_libvirt_gethostbyname4_r #12 0x00007f6e724631fc in gaih_inet #13 0x00007f6e72464697 in __GI_getaddrinfo #14 0x00007f6e71d19e81 in virGetHostnameImpl #15 0x00007f6e71d1a057 in virGetHostnameQuiet #16 0x00007f6e71d09936 in virLogOnceInit #17 0x00007f6e71d09952 in virLogOnce #18 0x00007f6e714b5829 in __pthread_once_slow #19 0x00007f6e71d16e7d in virOnce #20 0x00007f6e71d0997c in virLogInitialize #21 0x00007f6e71d0a09a in virLogVMessage #22 0x00007f6e71d09ffd in virLogMessage #23 0x00007f6e71d0db22 in virObjectNew #24 0x00007f6e71d0dbf1 in virObjectLockableNew #25 0x00007f6e71d0d3e5 in virMacMapNew #26 0x00007f6e71cdc50a in findLease #27 0x00007f6e71cdc839 in _nss_libvirt_gethostbyname3_r #28 0x00007f6e71cdc724 in _nss_libvirt_gethostbyname2_r #29 0x00007f6e7248f72f in __gethostbyname2_r #30 0x00007f6e7248f494 in gethostbyname2 #31 0x000056348c30c36d in hosts_keys #32 0x000056348c30b7d2 in main Fortunately the extra stuff virGetHostname does is totally irrelevant to the needs of the logging code, so we can just inline a call to the native hostname() syscall directly. 2018-02-12 Peter Krempa qemu: command: Extract formatting of floppy related stuff into a helper The floppy command formatting is special-cased since it does not directly translate to a single '-device' argument. Move the code from qemuBuildDiskDriveCommandLine to a new helper function so that all the related code is together. 2018-02-12 Chen Hanxiao qemu: don't leak in qemuGetDHCPInterfaces when failing to alloc We forgot to free alloced mem when failed to dup ifname or macaddr. Also use VIR_STEAL_PTR to simplify codes. 2018-02-09 Chen Hanxiao util: virnetlink: Fix the parameter description of functions Some of function comments don't have the right named parameters and others are not consistent with the description alignment. This patch fixes this. 2018-02-09 Andrea Bolognani qemu: Remove redundancy from qemuBuildControllerDevStr() Several PCI controllers have the same options, and thus can be handled together. Reviewed-by: John Ferlan 2018-02-09 Andrea Bolognani qemu: Error out on invalid pci-root controller model name This is a hard error, and should be handled as such. Introduced in 24614760228b. Reviewed-by: John Ferlan 2018-02-09 Andrea Bolognani qemu: Fix indentation in qemuDomainDeviceDefValidateControllerPCI() 2018-02-09 Daniel P. Berrangé rpc: export xdr_virNetMessageError for virnetmessagetest on Win32 The Win32 symbol export file format can't do wildcards, so none of the 'xdr_*' symbols are exported from the libvirt DLL. This doesn't matter generally since the RPC client is built into the DLL and we don't build libvirtd on Win32. The virnetmessagetest, however, does require xdr_virNetMessageError to be exported, so just do a hack for that. 2018-02-09 John Ferlan lockd: Install the admin sockets Commit id 'ce7ae55e' added support for the lockd admin socket, but forgot to add the socket to the make and spec files for installation purposes. Reviewed-by: Daniel P. Berrange 2018-02-09 John Ferlan logd: Install the admin sockets Commit id '85d45ff0' added support for the logd admin socket, but forgot to add the socket to the make and spec files for installation purposes. NB: Includes breaking up the long %systemd_ lists across multiple lines for ease of reading Reviewed-by: Daniel P. Berrange 2018-02-09 Daniel P. Berrange docs: document requirement to provide Signed-off-by lines for DCO Document that contributors are required to assert compliance with the Developers Certification of Origin 1.1, by providing Signed-off-by tags for all commit messages. The DCO is formally stating what we have long implicitly expected of contributors in terms of their legal rights to make the contribution. This puts the project in a stronger position should any questions around contributions be raised going forward in the future. 2018-02-09 Daniel P. Berrangé cfg: forbid includes of headers in network and storage drivers again Loadable drivers must never depend on each other. Over time some usage mistakenly crept in for the storage and network drivers, but now this is eliminated the syntax-check rules can enforce this separation once more. 2018-02-09 Daniel P. Berrangé build: passing the "-z defs" linker flag to prevent undefined symbols Undefined symbols are a bad thing in general because they can get resolved in unexpected ways at runtime if multiple sources provide the same symbol name. For example both glibc and libtirpc may provide XDR symbols and we want to ensure that we resolve to libtirpc if that's what we originally built against. The toolchain maintainers thus strongly recommend that all applications use the '-z defs' linker flag to prevent undefined symbols. This is shortly becoming part of the default linker flags for RPMs. As an added benefit this aligns Linux builds with Windows builds, where the linker has never permitted undefined symbols. 2018-02-09 Daniel P. Berrangé build: provide a AM_FLAGS_MOD for loadable modules Dynamic loadable modules all need a common set of linker flags -module -avoid-version $(AM_LDFLAGS) Bundle those up into a $(AM_LDFLAGS_MOD) to avoid repetition. 2018-02-09 Daniel P. Berrangé build: explicitly link all modules with libvirt.so The dlopened modules we currently build all use various symbols from libvirt.so, but don't actually link to it. They rely on the libvirtd daemon re-exporting the libvirt.so symbols. This means that at the time the modules are linked, they contain a huge number of undefined symbols. It also means that these undefined symbols are not versioned, so despite us providing a LIBVIRT_PRIVATE_XXXX version that intentionally changes on every release, the loadable modules could actually be loaded into any libvirtd regardless of version. This change explicitly links all modules against libvirt.so so that they don't rely on the re-export behave and can be fully resolved at build time. This will give us a stronger guarantee modules will actually be loadable at runtime and that we're using modules from the matched build. 2018-02-09 Daniel P. Berrangé storage: export virStoragePoolLookupByTargetPath as a public API The storagePoolLookupByTargetPath() method in the storage driver is used by the QEMU driver during block migration. If there's a valid use case for this in the QEMU driver, then external apps likely have similar needs. Exposing it in the public API removes the direct dependancy from the QEMU driver to the storage driver. conf: move virStorageTranslateDiskSourcePool into domain conf The virStorageTranslateDiskSourcePool method modifies a virDomainDiskDef to resolve any storage pool reference. For some reason this was added into the storage driver code, despite working entirely in terms of the public APIs. Move it into the domain conf file and rename it to match the object it modifies. network: remove conditional declarations The networkDnsmasqConfContents() method is only used by the test suite and that's only built with WITH_NETWORK is set. So there is no longer any reason to conditionalize the declaration of this method. 2018-02-09 Daniel P. Berrangé conf: expand network device callbacks to cover resolving NIC type Currently the QEMU driver will call directly into the network driver impl to modify resolve the atual type of NICs with type=network. It has todo this before it has allocated the actual NIC. This introduces a callback system to allow us to decouple the QEMU driver from the network driver. This is a short term step, as it ought to be possible to achieve the same end goal by simply querying XML via the public network API. The QEMU code in question though, has no virConnectPtr conveniently available at this time. 2018-02-09 Daniel P. Berrangé qemu: replace networkGetNetworkAddress with public API calls The QEMU driver calls into the network driver to get the first IP address of the network. This information is readily available via the formal public API by fetching the XML doc and then parsing it. conf: expand network device callbacks to cover bandwidth updates Currently the QEMU driver will call directly into the network driver impl to modify network device bandwidth for interfaces with type=network. This introduces a callback system to allow us to decouple the QEMU driver from the network driver. conf: introduce callback registration for domain net device allocation Currently virt drivers will call directly into the network driver impl to allocate domain interface devices where type=network. This introduces a callback system to allow us to decouple the virt drivers from the network driver. 2018-02-09 Daniel P. Berrangé build: link libvirt_lxc against libvirt.so Rather than static linking in various of the helper libraries to libvirt_lxc, just link against the main libvirt.so. This is more memory and time efficient because it will already be cached in memory and sharable between processes. CAPNG flags need adding because the LXC code directly calls various libcapng APIs and no longer inherits the CAPNG flags via the statically linked .a libs. 2018-02-09 Daniel P. Berrangé rpc: don't link in second copy of RPC code to libvirtd & lockd plugin The libvirt_driver_remote.la static library is linked into the libvirt.so dynamic library, providing both the generic RPC layer code and the remote protocol client driver. The libvirtd daemon the itself links to libvirt_driver_remote.la, in order to get access to the generic RPC layer code and the XDR functions for the remote driver. This means we get multiple copies of the same code in libvirtd, one direct and one indirect via libvirt.so. The same mistake affects the lockd plugin. The libvirtd daemon should instead just link aganist the generic RPC layer code that's in libvirt.so. This is easily doable if we add exports for the few symbols we've previously missed, and wildcard export xdr_* to expose the auto-generated XDR marshallers. 2018-02-09 Daniel P. Berrangé storage: move storage file backend framework into util directory The QEMU driver loadable module needs to be able to resolve all ELF symbols it references against libvirt.so. Some of its symbols can only be resolved against the storage_driver.so loadable module which creates a hard dependancy between them. By moving the storage file backend framework into the util directory, this gets included directly in the libvirt.so library. The actual backend implementations are still done as loadable modules, so this doesn't re-add deps on gluster libraries. 2018-02-09 Daniel P. Berrangé storage: extract storage file backend from main storage driver backend The storage driver backends are serving the public storage pools API, while the storage file backends are serving the internal QEMU driver and / or libvirt utility code. To prep for moving this storage file backend framework into the utility code, split out the backend definitions. 2018-02-08 Tiago M. Vieira fix regex to check CN from server certificate Currently when the script validates the PKI files and the certificate 'Subject:' field contains RDNs after the Common Name (CN), these values are also included, creating a false result that the CN is not correct. A small change to the sed regex fixes this issue, by extracting only the value for CN and nothing else. The regex is replaced with the exact same regex used to extract the CN value from the client certificate. 2018-02-08 Christian Ehrhardt AUTHORS: Add myself to the list of committers 2018-02-07 Andrea Bolognani util: Fix syntax-check Broken by 759b4d1b0fe5f4d84d98b99153dfa7ac289dd167. 2018-02-07 Lubomir Rintel virlog: determine the hostname on startup CVE-2018-6764 At later point it might not be possible or even safe to use getaddrinfo(). It can in turn result in a load of NSS module. Notably, on a LXC container startup we may find ourselves with the guest filesystem already having replaced the host one. Loading a NSS module from the guest tree would allow a malicous guest to escape the confinement of its container environment because libvirt will not yet have locked it down. 2018-02-07 Viktor Mihajlovski qemu: Limit refresh of CPU halted state to s390 Refreshing the halted state can cause VM performance issues. Since s390 is currently the only architecture with a known interest in the halted state, we're avoiding to call QEMU on other platforms. 2018-02-07 Peter Krempa qemu: domain: Store vcpu halted state as a tristate Since it may be possible that the state is unknown in some cases we should store it as a tristate so that other code using it can determine whether the state was updated. qemu: Remove unused 'cpuhalted' argument from qemuDomainHelperGetVcpus The halted state is no longer extracted using this helper so the argument can be removed. qemu: driver: Extract vcpu halted state directly Don't extract the halted state into a separate array, but rater access the vcpu structures directly. We still need to call the vcpu helper to retrieve the performance statistics though. 2018-02-07 Michal Privoznik conf: Check for NUMA distances in validity check NUMA distances are part of guest ABI (guests can read it directly!) and therefore as such shouldn't change throughout the lifetime of domain. 2018-02-06 Guido Günther apparmor: allow libvirt to send term signal to unconfined Otherwise stopping domains with qemu://session fails like [164012.338157] audit: type=1400 audit(1516202208.784:99): apparmor="DENIED" operation="signal" profile="/usr/sbin/libvirtd" pid=18835 comm="libvirtd" requested_mask="send" denied_mask="send" signal=term peer="unconfined" 2018-02-06 Shivaprasad G Bhat virt-aa-helper: Set the supported features The virt-aa-helper fails to parse the xmls with the memory/cpu hotplug features or user assigned aliases. Set the features in xmlopt->config for the parsing to succeed. Tested-by: Christian Ehrhardt Reviewed-by: Christian Ehrhardt 2018-02-06 John Ferlan qemu: Initialize @priv in qemuDomainCoreDumpWithFormat Fix for a CI build failure docs: Add news article for query memory-only dump processing percentage Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Allow showing the dump progress for memory only dump https://bugzilla.redhat.com/show_bug.cgi?id=916061 If the QEMU version running is new enough (based on the DUMP_COMPLETED event), then we can add a 'detach' boolean to the dump-guest-memory command in order to tell QEMU to run in a thread. This ensures that we don't lock out other commands while the potentially long running dump memory is completed. This allows the usage of a qemuDumpWaitForCompletion which will wait for the event while the qemuDomainGetJobInfoDumpStats can be used via qemuDomainGetJobInfo in order to query QEMU to determine how far along the job is. Now that we have a true async job, we'll only set the dump_memory_only flag only when @detach=false; otherwise, we note that the job is a for stats dump this allows the opposite end for job info to determine what to copy. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Add new parameter to qemuMonitorDumpToFd Add a @detach parameter to the API in order allow running the QEMU code as a thread. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Introduce qemuDomainGetJobInfoDumpStats Add an API to allow fetching the memory only dump statistics for a job via the qemuDomainGetJobInfo API. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Introduce qemuMonitor[JSON]QueryDump Add the query-dump API's in order to allow the dump-guest-memory to be used to monitor progress. This will use the dump stats extraction helper to fill a return buffer. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Introduce qemuProcessHandleDumpCompleted Handle a DUMP_COMPLETED event processing the status, stats, and error string. Use the @status in order to copy the error that was generated whilst processing the @stats data. If an error was provided by QEMU, then use that instead. If there's no async job, we can just ignore the data. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Add support for DUMP_COMPLETED event The event will be fired when the domain memory only dump completes. Fill in a return buffer to store/pass along the dump statistics that will be eventually shared by a query-dump command. Also pass along the status of the filling and any possible error received. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_MEMDUMP Define the qemuMonitorDumpStats as a new job JobStatsType to handle being able to get memory dump statistics. For now do nothing with the new TYPE_MEMDUMP. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Introduce QEMU_DOMAIN_JOB_STATS_TYPE_SAVEDUMP Add a TYPE_SAVEDUMP so that when coalescing stats for a save or dump we don't needlessly try to get the mirror stats for a migration. Other conditions can still use MIGRATION and SAVEDUMP interchangably including usage of the @migStats field to fetch/store the data. Reviewed-by: Jiri Denemark 2018-02-06 John Ferlan qemu: Convert jobInfo stats into a union Convert the stats field in _qemuDomainJobInfo to be a union. This will allow for the collection of various different types of stats in the same field. When starting the async job that will end up being used for stats, set the @statsType value appropriately. The @mirrorStats are special and are used with stats.mig in order to generate the returned job stats for a migration. Using the NONE should avoid the possibility that some random async job would try to return stats for migration even though a migration is not in progress. For now a migration and a save job will use the same statsType Reviewed-by: Jiri Denemark 2018-02-05 Peter Krempa util: bitmap: Note that shrinking the bitmap requires clearing of unused bits Note the fact that the unused portion of the last element in the bitmap needs to be cleared, since we use functions which process only full-size elements and don't really deal with individual bits. 2018-02-05 Peter Krempa util: bitmap: Use VIR_SHRINK_N in virBitmapShrink The function only reduces the size of the bitmap thus we can use the appropriate shrinking function which also does not have any return value. Since virBitmapShrink now does not return any value callers need to be fixed as well. 2018-02-05 Peter Krempa util: bitmap: Fix value of 'map_alloc' when shrinking bitmap The virBitmap code uses VIR_RESIZE_N to do quadratic scaling, which means that along with the number of requested map elements we also need to keep the number of actually allocated elements for the scaling algorithm to work properly. The shrinking code did not fix 'map_alloc' thus virResizeN might actually not expand the bitmap properly after called on a previously shrunk bitmap. 2018-02-05 Peter Krempa util: bitmap: Add comments for functions which don't have them virBitmap code is thoroughly documented. Add docs for the few functions missing them. util: bitmap: Fix function formatting and spacing util: bitmap: Rename 'max_bit' to 'nbits' 'max_bit' is misleading as the value is set to the first invalid bit as it's used as the number of bits in the bitmap. Rename it to a more descriptive name. 2018-02-05 Martin Kletzander qemu: Refresh capabilities when creating resctrl allocation Since one of the things in capabilities (info from resctrl updated with data about caches) can be change on the system by remounting the /sys/fs/resctrl with different options, the capabilities need to be refreshed. There is a better fix in the works, but it's going to be way bigger than this (hence the XXX note there), so for the time being let's workaround this. And in order not to slow down the domain starting, only get the capabilities if there are any cachetunes. Relates-to: https://bugzilla.redhat.com/show_bug.cgi?id=1540780 2018-02-05 Martin Kletzander util: Check if kernel-provided info is consistent with itself Just in case someone re-mounted /sys/fs/resctrl with different mount options (cdp), add a check here. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540780 2018-02-05 Marc Hartmayer qemu: Add and use qemuProcessEventFree for freeing qemuProcessEvents Add and use qemuProcessEventFree for freeing qemuProcessEvents. This is less error-prone as the compiler can help us make sure that for every new enumeration value of qemuProcessEventType the qemuProcessEventFree function has to be adapted. All process*Event functions are *only* called by qemuProcessHandleEvent and this function does the freeing by itself with qemuProcessEventFree. This means that an explicit freeing of processEvent->data is no longer required in each process*Event handler. The effectiveness of this change is also demonstrated by the fact that it fixes a memory leak of the panic info data in qemuProcessHandleGuestPanic. Reported-by: Wang Dong Reviewed-by: Boris Fiuczynski 2018-02-05 Marc Hartmayer qemu: Use the return value of virObjectRef directly Use the return value of virObjectRef directly. This way, it's easier for another reader to identify the reason why the additional reference is required. Reviewed-by: Boris Fiuczynski Reviewed-by: Bjoern Walk 2018-02-03 John Ferlan netserver: Remove ServiceToggle during ServerDispose No sense in calling ServiceToggle for all nservices during ServiceDispose since ServerClose calls ServiceClose which removes the IOCallback that's being toggled via ServiceToggle. Reviewed-by: Erik Skultety 2018-02-02 John Ferlan qemu: Add dump completed event to the capabilities Add the DUMP_COMPLETED check to the capabilities. This is the mechanism used to determine whether the dump-guest-memory command can support the "-detach" option and thus be able to wait on the event and allow for a query of the progress of the dump. Reviewed-by: Jiri Denemark 2018-02-02 John Ferlan qemu: Introduce qemuDomainGetJobInfoMigrationStats Extract out the parts of qemuDomainGetJobStatsInternal that get the migration stats. We're about to add the ability to get just dump information. Reviewed-by: Jiri Denemark 2018-02-02 Laine Stump vbox: fix SEGV during dumpxml of a serial port commit 77a12987a48 changed the "virDomainChrSourceDef source" inside virDomainChrDef to "virDomainChrSourceDefPtr source", and started allocating source inside virDomainChrDefNew(), but vboxDumpSerial() was allocating a virDomainChrDef with a simple VIR_ALLOC() (i.e. never calling virDomainChrDefNew()), so source was never initialized, leading to a SEGV any time a serial port was present. The same problem was created in vboxDumpParallel(). This patch changes vboxDumpSerial() and vboxDumpParallel() to use virDomainChrDefNew() instead of VIR_ALLOC(), and changes both of those functions to return an error if virDomainChrDef() (or any other allocation) fails. This resolves: https://bugzilla.redhat.com/1536649 2018-02-02 John Ferlan nwfilter: Remove unnecessary UUID comparison bypass Remove the unnecessary check as since commit id '46a811db07' it is not possible to add or alter a filter using the same name, but with a different UUID. NB: It's not required to provide a UUID for a filter by name, but if one is provided, then it must match the existing. If not provided, then one is generated during ParseXML processing. Reviewed-by: Laine Stump Reviewed-by: Stefan Berger 2018-02-02 John Ferlan util: Remove unnecessary initialization VIR_ALLOC will already initialize, so no need to do it again. 2018-02-02 Marc Hartmayer qemu: Use switch statement for address types in qemuBuildControllerDevStr Use a switch statement instead of if-else-if statements. Move the command line building of the iothread attribute into the common path as the SCSI controller attributes are already validated. 2018-02-02 John Ferlan qemu: Introduce qemuDomainDeviceDefValidateControllerSATA Move the SATA controller check from command line building to controller def validation. This includes copying the SATA skip check found in qemuBuildSkipController. 2018-02-02 John Ferlan qemu: Complete PCI command checks to controller def validate Move the qemuCaps checks over to qemuDomainControllerDefValidatePCI. This requires two test updates in order to set the correct capability bit for an xml2xml test as well as setting up the similar capability for the pseries memlocktest. 2018-02-02 John Ferlan qemu: Move more PCI command checks to controller def validate Excluding the qemuCaps checks, move the remainder of the checks that validate whether the PCI definition is valid or not into qemuDomainControllerDefValidatePCI. 2018-02-02 John Ferlan qemu: Move PCI command modelName TypeToString to controller def validate Similar to the checking the modelName vs. NAME_NONE, let's make the ModelNameTypeToString check more generic too within the checking done in controller validation (with the same ignore certain models. NB: We need to keep the ModelNameTypeToString fetch in command line validation since we use it, but at least we can assume it returns something valid now. 2018-02-02 John Ferlan qemu: Move PCI command modelName check to controller def validate Move the various modelName == NAME_NONE from the command line generation into domain controller validation. Also rather than have multiple cases with the same check, let's make the code more generic, but also note that it was the modelName option that caused the failure. We also have to be sure not to check the PCI models that we don't care about. For the remaining checks in command line building, we can use the field name in the error message to be more specific about what causes the failure. 2018-02-02 John Ferlan qemu: Use virDomainPCIControllerOpts in qemuBuildControllerDevStr Shorten up a few characters and reference the pciopts pointer 2018-02-02 Andrea Bolognani qemu: Add missing checks for pcie-root-port options We format the 'chassis' and 'port' properties on the QEMU command line later on, so we should make sure they've been set. 2018-02-02 John Ferlan qemu: Introduce qemuDomainDeviceDefValidateControllerPCI Move PCI validation checks out of qemu_command into the proper qemu_domain validation helper. Since there's a lot to move, we'll start slow by replicating the pcie-root and pci-root avoidance from qemuBuildSkipController and the first switch found in qemuBuildControllerDevStr. 2018-02-02 John Ferlan qemu: Introduce qemuDomainDeviceDefValidateControllerSCSI Move SCSI validation from qemu_command into qemu_domain. Rename/reorder the args in qemuCheckSCSIControllerIOThreads to match the caller as well as fixing up the comments to remove the previously removed qemuCaps arg. 2018-02-02 John Ferlan qemu: Add check for iothread attribute in validate controller Let's make sure that non SCSI virtio-scsi isn't used for any type other than a virtio-scsi controller. qemu: Adjust SCSI controller switch in qemuBuildControllerDevStr Modify the SCSI controller switch during command line building to account for all virDomainControllerModelSCSI types rather than using the default label. qemu: Move and rename qemuBuildCheckSCSIControllerModel Move to qemu_domain during the validation of controller options and rename qemuDomainCheckSCSIControllerModel. qemu: Introduce qemuDomainDeviceDefValidateControllerAttributes Move the checks that various attributes are not set on any controller other than SCSI controller using virtio-scsi model into the common controller validate checks. 2018-02-02 Martin Kletzander tests: Add test for properly removing cachetune entries Cachetune for unavailable vCPUs should be cleared the same way vcpupin and other things do, so let's add tests for it. 2018-02-02 Martin Kletzander util: Clear unused part of the map in virBitmapShrink Some of the other functions depend on the fact that unused bits and longs are always zero and it's less error-prone to clear it than fix the other functions. It's enough to zero out one piece of the map since we're calling realloc() to get rid of the rest (and updating map_len). Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540817 2018-02-02 Martin Kletzander util: Fix possible leak in virResctrlAllocMasksAssign Found by coverity. 2018-02-02 Daniel P. Berrangé rpc: fix non-NULL annotations when GNUTLS is disabled The position of various parameters changes depending on the WITH_GNUTLS macro. Reviewed-by: John Ferlan 2018-02-02 Daniel P. Berrangé rpc: assume private data callbacks are always non-NULL Since we annotate the APIs are having non-NULL parameters, we can remove the checks for NULL in the code too. Reviewed-by: John Ferlan 2018-02-02 Daniel P. Berrangé tests: validate private data / pre / post exec hooks for RPC APIs Validate that the virNetServer(Client) RPC APIs are processing the private data callbacks correctly by passing in non-NULL pointers. Reviewed-by: John Ferlan 2018-02-02 Michal Privoznik qemuDomainRemoveMemoryDevice: unlink() memory backing file https://bugzilla.redhat.com/show_bug.cgi?id=1461214 Since fec8f9c49af we try to use predictable file names for 'memory-backend-file' objects. But that made us provide full path to qemu when hot plugging the object while previously we provided merely a directory. But this makes qemu behave differently. If qemu sees a path terminated with a directory it calls mkstemp() and unlinks the file immediately. But if it sees full path it just calls open(path, O_CREAT ..); and never unlinks the file. Therefore it's up to libvirt to unlink the file and not leave it behind. 2018-02-02 Peter Krempa qemu: migration: Refresh device information after transferring state In my first approach in 4b480d10768c I overlooked the comment in qemuMigrationRunIncoming stating that during actual migration the qemuMigrationRunIncoming does not wait until the migration is complete but rather offloads that to the Finish phase of migration. This means that during actual migration qemuProcessRefreshState was called prior to qemu actually transferring the full state and thus the queries did not get the correct information. The approach worked only for restore, where we wait for the migration to finish during qemu startup. Fix the issue by calling qemuProcessRefreshState both from qemuProcessStart if there's no incomming migration and from qemuMigrationFinish so that the code actually works as expected. 2018-02-02 Michal Privoznik qemu: Expose rx/tx_queue_size in qemu.conf too In 2074ef6cd4a2 and c56cdf259 (and friends) we've added two attributes to virtio NICs: rx_queue_size and tx_queue_size. However, sysadmins might want to set these on per-host basis but don't necessarily have an access to domain XML (e.g. because they are generated by some other app). So let's expose them under qemu.conf (the settings from domain XML still take precedence as they are more specific ones). Reviewed-by: John Ferlan 2018-02-01 Jie Wang blockjob: Fix a error checking of blockjob status in some case Commit id 'bc444666f' added a check if the returned data buffer had an error, but failed to adjust the event from VIR_DOMAIN_BLOCK_JOB_COMPLETED to VIR_DOMAIN_BLOCK_JOB_FAILED in order to propagate an error such as "File descriptor in bad state" that may be returned from QEMU when both @offset and @len are set to 0 such as is the case when performing an async block job read on a read only filesystem. Reviewed-by: John Ferlan 2018-02-01 Chen Hanxiao news: Add VIR_ERR_DEVICE_MISSING change as improvements Reviewed-by: John Ferlan 2018-02-01 Chen Hanxiao qemu: Use VIR_ERR_DEVICE_MISSING for various coldplug messages Use the DEVICE_MISSING error code when helpers fail to find the requested device. This makes it easier for consumers to key off the error code rather than the error message. Reviewed-by: John Ferlan 2018-02-01 Chen Hanxiao qemu: Use VIR_ERR_DEVICE_MISSING for various hotplug messages Modify OPERATION_FAILED and INTERNAL_ERROR error codes to use DEVICE_MISSING instead for failures associated with the inability to find the device. This makes it easier for consumers to key off the error code rather than the error message. Reviewed-by: John Ferlan 2018-02-01 Chen Hanxiao qemu: Introduce VIR_ERR_DEVICE_MISSING Add new error code to be able to allow consumers (such as Nova) to be able to key of a specific error code rather than needing to search the error message." Reviewed-by: John Ferlan 2018-02-01 Chen Hanxiao qemu: Add some more details for hotplug errors when device not found More proper/detail error messages updated. Reviewed-by: John Ferlan 2018-02-01 Andrea Bolognani conf: Small indentation and coding style fixes 2018-01-31 Daniel P. Berrangé admin: set XDR_CFLAGS when building admin server The XDR_CFLAGS variable is needed when the XDR headers are not part of the stock glibc install storage: remove virConnectPtr from all backend functions Now that we can open connections to the secondary drivers on demand, there is no need to pass a virConnectPtr into all the backend functions. storage: open nodedev driver connection at time of use Instead of passing around a virConnectPtr object, just open a connection to the nodedev driver at time of use. Opening connections on demand will be beneficial when the nodedev driver is in a separate daemon. It also solves the problem that a number of callers just pass in a NULL connection today which prevents nodedev lookup working at all. storage: open secret driver connection at time of use Instead of passing around a virConnectPtr object, just open a connection to the secret driver at time of use. Opening connections on demand will be beneficial when the secret driver is in a separate daemon. It also solves the problem that a number of callers just pass in a NULL connection today which prevents secret lookup working at all. driver: add some helpers for opening secondary driver connections Various parts of libvirt will want to open connections to secondary drivers. The right URI to use will depend on the context, so rather than duplicating that logic in various places, use some helper APIs. This will also make it easier for us to later pre-open/cache connections to avoid repeated opening & closing the same connectiong during autostart. 2018-01-31 Daniel P. Berrangé secret: allow opening with secret:///system and secret:///session URIs Allow the possibility of opening a connection to only the secret driver, by defining secret:///system and secret:///session URIs and registering a fake hypervisor driver that supports them. The hypervisor drivers can now directly open a secret driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé nodedev: allow opening with nodedev:///system and nodedev:///session URIs Allow the possibility of opening a connection to only the nodedev driver, by defining nodedev:///system and nodedev:///session URIs and registering a fake hypervisor driver that supports them. The hypervisor drivers can now directly open a nodedev driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé interface: allow opening with interface:///system and interface:///session URIs Allow the possibility of opening a connection to only the interface driver, by defining interface:///system and interface:///session URIs and registering a fake hypervisor driver that supports them. The hypervisor drivers can now directly open a interface driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé nwfilter: allow opening with nwfilter:///system URI Allow the possibility of opening a connection to only the storage driver, by defining a nwfilter:///system URI and registering a fake hypervisor driver that supports it. The hypervisor drivers can now directly open a nwfilter driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé network: allow opening with network:///system and network:///session URIs Allow the possibility of opening a connection to only the network driver, by defining network:///system and network:///session URIs and registering a fake hypervisor driver that supports them. The hypervisor drivers can now directly open a network driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé network: move driver registration back to end of the file By convention the last thing in the driver.c files should be the driver callback table and function to register it. 2018-01-31 Daniel P. Berrangé storage: allow opening with storage:///system and storage:///session URIs Allow the possibility of opening a connection to only the storage driver, by defining storage:///system and storage:///session URIs and registering a fake hypervisor driver that supports them. The hypervisor drivers can now directly open a storage driver connection at time of need, instead of having to pass around a virConnectPtr through many functions. This will facilitate the later change to support separate daemons for each driver. 2018-01-31 Daniel P. Berrangé storage: move driver registration back to end of the file By convention the last thing in the driver.c files should be the driver callback table and function to register it. 2018-01-31 Daniel P. Berrange util: use union for sockaddr structs to avoid aliasing Some platforms/toolchains will complain about casting sockaddr_storage to sockaddr_un because it breaks strict aliasing rule ../../src/util/virutil.c: In function 'virGetUNIXSocketPath': ../../src/util/virutil.c:2005: error: dereferencing pointer 'un' does break strict-aliasing rules [-Wstrict-aliasing] Change the code to use a union, in the same way that the virsocketaddr.h header does. 2018-01-31 Daniel P. Berrangé tools: unlink bash completion files before symlinking "ln" will not replace an existing symlink, so if you run 'make install' twice, the second time will get an error: ln: failed to create symbolic link 'virsh': File exists We must always remove the symlink target first. 2018-01-31 Daniel P. Berrangé Fixed virGetUNIXSocketPath stub on Win32 The _() macro was not terminated and an argument needs to be marked as unused. 2018-01-31 John Ferlan qemu: Update qemuDomainFindSCSIControllerModel return Now that the controller model is updated during post parse callback, this code no longer needs to fetch the model based on the capabilities and can just return the model directly if the controller is found. Removal of @qemuCaps cascades through various callers which are now updated to not pass the capabilities. 2018-01-31 John Ferlan qemu: Reduce need to call qemuDomainGetSCSIControllerModel Now that post parse processing handles setting the SCSI controller model, there's no need to call qemuDomainGetSCSIControllerModel to get the "default controller" when building the command line controller string or when assigning the spaprvio address since the controller model value will already be filled in. conf: Allow configuration of implicit controller model When an implicit controller is added, the model is defined as -1 (IOW: undefined). So, if an implicit SCSI controller was added, can set the model to the default value if the underlying hypervisor supports it. qemu: Introduce qemuDomainSetSCSIControllerModel During post parse processing, let's force setting the controller model to default value if not already set for defined controllers (e.g. the non implicit ones). 2018-01-31 John Ferlan qemu: Fetch/save the default SCSI controller model during hotplug If we're going to add a controller to the domain, let's set the default SCSI model value if we cannot find another SCSI controller already present. NB: Requires updating the live output test data since the model will now be formatted. 2018-01-31 John Ferlan qemu: Introduce qemuDomainGetSCSIControllerModel Rename and rework qemuDomainSetSCSIControllerModel since we're really not setting the SCSI controller model. Instead the code is either returning the existing SCSI controller model value, the default value based on the capabilities, or -1 with the error set. 2018-01-31 John Ferlan qemu: Introduce qemuDomainFindSCSIControllerModel Rather than repeat multiple steps in order to find the SCSI controller model, let's combine them into one helper that will return either the model from the definition or the default model based on the capabilities. This patch adds an extra check/error that the controller that's being found actually exists. This just clarifies that the error was because the controller doesn't exist rather than the more generic error that we were unable to determine the model from qemuDomainSetSCSIControllerModel when a -1 was passed in and the capabilities were unable to find one. 2018-01-31 John Ferlan conf: Rework and rename virDomainDeviceFindControllerModel As it turns out virDomainDeviceFindControllerModel was only ever called for SCSI controllers using VIR_DOMAIN_CONTROLLER_TYPE_SCSI as a parameter. So rename to virDomainDeviceFindSCSIController and rather than return a model, let's return a virDomainControllerDefPtr to let the caller reference whatever it wants. 2018-01-31 John Ferlan qemu: Split qemuDomainSetSCSIControllerModel Rather than one function serving two purposes, let's split out the else condition which is checking whether the model can be used during command line building based on the capabilities. 2018-01-31 Michal Privoznik virUSBDeviceNew: Construct vroot path properly When starting an LXC container, the /dev entries are created under temp root (/var/run/libvirt/lxc/$name.dev), relabelled and then the root is pivoted. However, when it comes to USB devices which keep path to the device in the structure we need a way to override the default /dev/usb/... path because we want to work with the one under temp root. That's what @vroot argument is for in virUSBDeviceNew. However, what is being passed there is: vroot = /var/run/libvirt/lxc/lxc_0.dev/bus/usb Therefore, constructed path is wrong: dev->path = //var/run/libvirt/lxc/lxc_0.dev/bus/usb//dev/bus/usb/002/002 Reviewed-by: John Ferlan 2018-01-31 Daniel P. Berrangé Check for tabs in augeas files 2018-01-31 Daniel P. Berrange lockd: add support for admin protocol in virtlockd Add a virtlockd-admin-sock can serves the admin protocol for the virtlockd daemon and define a virtlockd:///{system,session} URI scheme for connecting to it. logd: add support for admin protocol in virtlogd Add a virtlogd-admin-sock can serves the admin protocol for the virtlogd daemon and define a virtlogd:///{system,session} URI scheme for connecting to it. 2018-01-31 Daniel P. Berrange rpc: refactor virNetServer setup for post-exec restarts With the current code it is neccessary to call virNetDaemonNewPostExecRestart() and then for each server that needs restarting you are supposed to call virNetDaemonAddSeverPostExecRestart() This is fine if there's only ever one server, but as soon as you have two servers it is impossible to use this design. The code has no idea which servers were recorded in the JSON state doc, nor in which order the hash table serialized its keys. So this patch changes things so that we only call virNetDaemonNewPostExecRestart() passing in a callback, which is invoked once for each server found int he JSON state doc. 2018-01-31 Daniel P. Berrange rpc: add method for checking if a named server exists It is not possible to blindly call virNetDaemonGetServer() because in a post-exec restart scenario, some servers may not exist and this method will pollute the error logs. rpc: annotate various parameters as being required to be non-NULL The server name and client data callbacks need to be non-NULL or the system will crash at various times. This is particularly bad when some of the crashes only occur post-exec restart. rpc: pass virNetServer to post-exec restart callback in typesafe manner The virNetServer class is passing a pointer to itself to the virNetServerClient as a 'void *' pointer. This is presumably due to fact that the virnetserverclient.h file doesn't see the virNetServerPtr typedef. The typedef is easily movable though, which lets us get typesafe parameter passing, removing the confusion of passing two distinct 'void *' pointers to one method. rpc: clarify "void *" values passed to client callbacks util: add virGetUNIXSocketPath helper When receiving multiple socket FDs from systemd, it is critical to know what socket address each corresponds to so we can setup the right protocols on each. admin: add support for post-exec restart callbacks We don't have any per-client private data we need to persist, but the RPC infrastructure requires that we provide the callbacks and serialize an empty JSON object. This makes us future proof going forwards. libvirtd: rename virNetServerClient callback impls to match type names admin: move admins server impl/dispatch into src/admin directory The admin server functionality is a generic concept that should be wired up into all libvirt daemons, but is currently integrated with the libvirtd code. Move it all into the src/admin directory to prepare for broader reuse. 2018-01-31 Michal Privoznik tools: Make symlinks to vsh bash-completion script The bash-completion project documents that only those scripts from $BASH_COMPLETIONS_DIR that share name with the current command for which was hit are loaded [1]. This means, that vsh script we have there is not loaded. We have to create symlinks for virsh and virt-admin. At the same time, we have to create new RPM package because virt-admin and client packages are independent. That means we cannot place the vsh script in either of them. What we can do is to have a different package that contains the completion script and then virt-admin and client packages contain only the symlink and require the bash-completion package. 1: https://github.com/scop/bash-completion#faq 2018-01-31 Michal Privoznik virsh: Offer only persistent domains for autostart The 'autostart' command accepts only persistent domains. Make the completer return only those. virshDomainNameCompleter: Prune accepted flags Only a small subset of VIR_CONNECT_LIST_DOMAINS_* flags are actually used for this completer. Remove the unused ones. Note that this is unrelated to other commands using VIR_CONNECT_LIST_DOMAINS_* (i.e. cmdList) as this commit targets the completer only and nothing else. 2018-01-31 Martin Kletzander qemu: Restore resctrl alloc data after restart During reconnect we need to reconstruct the paths of all cachetunes so that they get cleaned up when the domain is stopped. util: Extract path formatting into virResctrlAllocDeterminePath We can use this from more places later, so just a future code de-duplication. qemu: Restore machinename even without cgroups The virresctrl will use this as well and we need to have that info after restart to properly clean up /sys/fs/resctrl. 2018-01-31 Martin Kletzander util: Don't overwrite mask in virResctrlAllocFindUnused Due to confusing naming the pointer to the mask got copied which must not happen, so use UpdateMask instead of SetMask. That also means we can get completely rid of SetMask. Also don't clear the free bits since it is not used again (leftover from previous versions). 2018-01-31 Martin Kletzander util: Use default group's mask for unspecified resctrl allocations Introduce virResctrlAllocCopyMasks() and use that to initially copy the default group schemata to the allocation before reserving any parts of the cache. The reason for this is that when new group is created the schemata will have unknown data in it. If there was previously group with the same CLoS ID, it will have the previous valies, if not it will have all bits set. And we need to set all unspecified (in the XML) allocations to the same one as the default group. Some non-Linux functions now need to be made public due to this change. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1289368 2018-01-31 Martin Kletzander util: Add helpers for getting resctrl group allocs 2018-01-31 Peter Krempa util: storage: Parse 'lun' for iSCSI protocol from JSON as string or number While the QEMU QAPI schema describes 'lun' as a number, the code dealing with JSON strings does not strictly adhere to this schema and thus formats the number back as a string. Use the new helper to retrieve both possibilities. Note that the formatting code is okay and qemu will accept it as an int. Tweak also one of the test strings to verify that both formats work with libvirt. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540290 2018-01-31 Peter Krempa util: json: Add helper to return string or number properties as string The helper is useful in cases when the JSON we have to parse may contain one of the two due to historical reasons and the number value itself would be stored as a string. 2018-01-30 Andrea Bolognani conf: Use correct attribute name in error message The feature has an attribute named 'policy', but the error message mentioned the non-existing 'state' attribute instead. 2018-01-30 Peter Krempa storage: util: Properly ignore errors when backing volume is inaccessible Commit 000e9504559 tried to fix improper bracketing when refreshing disk volume stats for a backing volume. Unfortunately the condition is still wrong as in cases as the backing store being inaccessible storageBackendUpdateVolTargetInfo returns -2 if instructed to ignore errors. The condition does not take this into account. Dumping XML of a volume which has inacessible backing store would then result into: # virsh vol-dumpxml http.img --pool default error: An error occurred, but the cause is unknown Properly ignore -2 for backing volumes. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540022 2018-01-29 Martin Kletzander util: Don't check if entries under /sys/fs/resctrl/(info/) are directories We are skipping non-directories under /sys/fs/resctrl/(info/) since those are not interesting for us. However in tests it can sometimes happen that ent->d_type is 0 instead of 4 (DT_DIR) for directories. I've seen it fail on two machines. Different machines, different systems, I cannot reproduce it even using the same setup. So one of the ways how to work around this is call stat() on it. The other one is not checking if it is a directory since we'll find out eventually when we want to read some files underneath it. 2018-01-29 Martin Kletzander util: Remove unused variable in virResctrlGetInfo util: Make it possible for virResctrlAllocSetMask to replace existing mask This wil be used in the future, but it makes sense for now as well. It makes sure there is no mask leftover that would leak. util: Use "resctrl" instead of "resctrlfs" spelling Pointed out during review on one or two places, but it actually appears in lot more places. So let's be consistent. util: Check for empty allocation instead of just NULL pointer When working on the CAT series one of the changes was that the pointer got allocated in another part of the code, even when resctrl was not available on the host system. However this one particular place neglected that so it needs to be fixed in order to get the proper error message when requesting on HW with no support for it. 2018-01-29 Erik Skultety build: Fix broken build on FreeBSD and OSX after recent nodedev series Commits f83c7c88 and 6eb1f2b9 broke the build on FreeBSD and OSX because of symbols being undefined for those platforms. 2018-01-29 John Ferlan qemu: Fix memory leak in processGuestPanicEvent After processing the processEvent->data for a qemuProcessEventHandler callout, it's expected that the called processEvent->eventType helper will perform the proper free on the data field. In this case it's a qemuMonitorEventPanicInfoPtr. 2018-01-29 Kashyap Chamarthy docs: formatdomain: Document the CPU feature 'name' attribute Currently, the CPU feature 'name' XML attribute, as in: [...] IvyBridge Intel [...] isn't explicitly documented in formatdomain.html. Document it now. 2018-01-29 Erik Skultety conf: nodedev: Update PCI mdev capabilities dynamically Just like SRIOV, a PCI device is only capable of the mediated devices framework when it's bound to the vendor native driver, thus if a driver change occurs, e.g. vendor_native->vfio, we need to refresh some of the device's capabilities to reflect the reality, mdev included. Suggested-by: Wu Zongyong 2018-01-29 Erik Skultety conf: Replace usage of virNodeDevCapMdevType with virMediatedDeviceType Now that we have all the building blocks in place, switch the nodedev driver to use the "new" virMediatedDeviceType type instead of the "old" virNodeDevCapMdevType one. nodedev: udev: Drop the unused mdev type helpers These are not necessary anymore, since these are going to be shadowed by the helpers provided by util/virmdev.c module. util: pci: Introduce virPCIGetMdevTypes helper This is a replacement for the existing udevPCIGetMdevTypesCap which is static to the udev backend. This simple helper constructs the sysfs path from the device's base path for each mdev type and queries the corresponding attributes of that type. util: mdev: Introduce virMediatedDeviceTypeReadAttrs getter This should serve as a replacement for the existing udevFillMdevType which is responsible for fetching the device type's attributes from the sysfs interface. The problem with the existing solution is that it's tied to the udev backend. util: mdev: Introduce virMediatedDeviceType structure This is later going to replace the existing virNodeDevCapMdevType, since: 1) it's going to couple related stuff in a single module 2) util is supposed to contain helpers that are widely accessible across the whole repository. util: mdev: Drop some unused symbols/includes from the header There were some leftovers from early development which never got used. 2018-01-29 Erik Skultety conf: nodedev: Refresh capabilities before touching them Most of them are static, however in case of PCI and SCSI_HOST devices, the nested capabilities can change dynamically, e.g. due to a driver change (from host_pci_driver -> vfio_pci). Suggested-by: Wu Zongyong 2018-01-29 Erik Skultety nodedev: Introduce virNodeDeviceCapsListExport Whether asking for a number of capabilities supported by a device or listing them, it's handled essentially by a copy-paste code, so extract the common stuff into this new helper which also updates all capabilities just before touching them. nodedev: Export nodeDeviceUpdateCaps from node_device_conf.c Since we moved the helpers from nodedev driver to src/conf, the actual 'update' function using those helpers should be moved as well so that we don't need to call back into the driver. nodedev: Move the sysfs-related cap handling to node_device_conf.c The capabilities are defined/parsed/formatted/queried from this module, no reason for 'update' not being part of the module as well. This also involves some module-specific prefix changes. This patch also drops the node_device_linux_sysfs module from the repo since: a) it only contained the capability handlers we just moved b) it's only linked with the driver (by design) and thus unreachable to other modules c) we touch sysfs across all the src/util modules so the module being deleted hasn't been serving its original intention for some time already. nodedev: Drop the nodeDeviceSysfsGetSCSIHostCaps wrapper We can call directly the virNodeDeviceGetSCSIHostCaps helper instead. conf: nodedev: Convert virNodeDevObjHasCapStr to a simple wrapper This patch drops the capability matching redundancy by simply converting the string input to our internal types which are then in turn used for the actual capability matching. conf: nodedev: Rename virNodeDeviceCapMatch to virNodeDevObjHasCap We currently have 2 methods that do the capability matching. This should be condensed to a single function and all the derivates should just call into that using a proper type conversion. conf: nodedev: Rename virNodeDevObjHasCap to virNodeDevObjHasCapStr We currently have 2 methods that do the capability matching. This should be condensed to a single function and all the derivates should just call into that using a proper type conversion. 2018-01-29 Julio Faracco test: Implementing testDomainRename(). There is no method to rename inactive domains for test driver. After this patch, we can rename the domains using 'domrename'. virsh# domrename test anothertest Domain successfully renamed 2018-01-29 Andrea Bolognani tests: Clean up GIC test cases These test cases are supposed to verify GIC support works as expected, and shouldn't concern themselves with other features; we can trim them down significantly, and make them less likely to need updating after unrelated changes. 2018-01-26 ZhiPeng Lu vhost-user: add support reconnect for vhost-user ports For vhost-user ports, Open vSwitch acts as the server and QEMU the client. When OVS crashes or restarts, the QEMU process should be reconnected to OVS. 2018-01-25 John Ferlan docs: Add missing element encryption description term entry Missed adding the "encryption" description term entry to the list of possible sub-elements for disk source. The description details were there, just not the tag. 2018-01-25 Daniel P. Berrangé resctl: stub out functions with Linux-only APIs used The flock() function and d_type field in struct dirent are not portable to the mingw platform. 2018-01-25 Daniel P. Berrange rpc: fix race sending and encoding sasl data The virNetSocketWriteSASL method has to encode the buffer it is given and then write it to the underlying socket. This write is not guaranteed to send the full amount of data that was encoded by SASL. We cache the SASL encoded data so that on the next invocation of virNetSocketWriteSASL we carry on sending it. The subtle problem is that the 'len' value passed into virNetSocketWriteSASL on the 2nd call may be larger than the original value. So when we've completed sending the SASL encoded data we previously cached, we must return the original length we encoded, not the new length. This flaw means we could potentially have been discarded queued data without sending it. This would have exhibited itself as a libvirt client never receiving the reply to a method it invokes, async events silently going missing, or worse stream data silently getting dropped. For this to be a problem libvirtd would have to be queued data to send to the client, while at the same time the TCP socket send buffer is full (due to a very slow client). This is quite unlikely so if this bug was ever triggered by a real world user it would be almost impossible to reproduce or diagnose, if indeed it was ever noticed at all. Reviewed-by: John Ferlan 2018-01-25 Jim Fehlig libxl: resume lock process after failed migration During migration, the lock process is paused in the perform phase but not resumed if there is a subsequent failure, leaving the locked resource unprotected. The perform phase itself can fail, in which case the lock process should be resumed before returning from perform. The finish phase could also fail on the destination host, in which case the migration is canceled in the confirm phase and the VM is resumed. The lock process needs to be resumed there as well. 2018-01-25 Martin Kletzander tests: Clean up and modify some vircaps2xmldata Basically the `cpus` and `tasks` files are not needed, and I've witnessed on a real system that the schemata file may have spaces prepended to a line, so let's adjust at least one test so that it reflects what can happen. Also `000` allocation is invalid and a full mask means it's all free. So adjust for that too. docs: Add CAT (resctrl) support into news.xml 2018-01-25 Martin Kletzander qemu: Add support for resctrl We've been building up to this. This adds support for cputune/cachetune settings for domains in the QEMU driver. The addition into qemuProcessSetupVcpu() automatically adds support for hotplug. For hot-unplug we need to remove the allocation only if all the vCPUs were unplugged. But since the threads are left running, we can't really do much about it now. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1289368 2018-01-25 Martin Kletzander tests: Add virresctrltest This test initializes capabilities from vircaps2xmldata (since it exists there already) and then requests list of free bitmaps (all unallocated space) from virresctrl.c Desirable outputs are saved in virresctrldata. 2018-01-25 Martin Kletzander conf: Add support for cputune/cachetune More info in the documentation, this is basically the XML parsing/formatting support, schemas, tests and documentation for the new cputune/cachetune element that will get used by following patches. 2018-01-25 Martin Kletzander resctrl: Add functions to work with resctrl allocations With this commit we finally have a way to read and manipulate basic resctrl settings. Locking is done only on exposed functions that read/write from/to resctrlfs. Not in functions that are exposed in virresctrlpriv.h as those are only supposed to be used from tests. More information about how resctrl works: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/x86/intel_rdt_ui.txt 2018-01-25 Martin Kletzander fixup_resctrlinfo util: Remove now-unneeded resctrl functions conf: Use virResctrlInfo in capabilities util: Add virResctrlInfo This will make the current functions obsolete and it will provide more information to the virresctrl module so that it can be used later. Rename virResctrlInfo to virResctrlInfoPerCache Just to ease the review of following patches. 2018-01-25 Erik Skultety vsh: Cmd aliases lookups should return results for the aliased command Unfortunately, we have a number of aliases in virsh and even though these are not visible any more, we have to support them. The problem is that when trying to print help for the alias, we get SIGSEGV because there isn't any @def structure anymore and we need to query the command being aliased instead. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1538570 2018-01-25 Erik Skultety vsh: Drop redundant definition searches from vshCmd{def,Grp}Help These helpers are called from a single place only - cmdHelp wrapper and just before the wrapper invokes the helpers, it performs the search, either for command group or for the command itself, except the result is discarded and the helper therefore needs to do it again. Drop this inefficient handling and pass the @def structure rather than a name, thus preventing the helper from needing to perform the search again. 2018-01-25 Daniel P. Berrange qemu: add support for generating SMBIOS OEM strings command line This wires up the previously added OEM strings XML schema to be able to generate comamnd line args for QEMU. This requires QEMU >= 2.12 release containing this patch: commit 2d6dcbf93fb01b4a7f45a93d276d4d74b16392dd Author: Daniel P. Berrange Date: Sat Oct 28 21:51:36 2017 +0100 smbios: support setting OEM strings table Reviewed-by: John Ferlan 2018-01-25 Daniel P. Berrange conf: add support for setting OEM strings SMBIOS data fields The OEM strings table in SMBIOS allows the vendor to pass arbitrary strings into the guest OS. This can be used as a way to pass data to an application like cloud-init, or potentially as an alternative to the kernel command line for OS installers where you can't modify the install ISO image to change the kernel args. As an example, consider if cloud-init and anaconda supported OEM strings you could use something like cloud-init:ds=nocloud-net;s=http://10.10.0.1:8000/ anaconda:method=http://dl.fedoraproject.org/pub/fedora/linux/releases/25/x86_64/os use of a application specific prefix as illustrated above is recommended, but not mandated, so that an app can reliably identify which of the many OEM strings are targetted at it. Reviewed-by: John Ferlan 2018-01-25 Shaohe Feng cpu: Add support for al57 Intel features We can start qemu with a "cpu,+la57" to set 57-bit vitrual address space. So VM can be aware that it need to enable 5-level paging. Corresponding QEMU commits: al57 6c7c3c21f95dd9af8a0691c0dd29b07247984122 2018-01-25 Daniel P. Berrangé mailmap: set preferred spelling for my name 2018-01-25 Daniel P. Berrange libxl: add explicit linkage to xenstore library Since commit eee7bd4ecb5a3e83ed47cc89bf0dc4866b4858af Author: Joao Martins Date: Tue Jul 26 00:45:14 2016 +0100 libxl: implement virDomainBlockStats Introduce initial support for domainBlockStats API the libxl driver calls a couple of xenstore APIs, so it must explicitly link to this library rather than rely on indirect linkage via libxl or other xen libraries. Reviewed-by: Jim Fehlig 2018-01-24 Michal Privoznik docs: Mention just implemented completers Reviewed-by: John Ferlan virsh: Introduce virshSnapshotNameCompleter Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshSecretUUIDCompleter This is a slight change from previous patches since virSecret does not have a name only UUID strings. Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshNWFilterNameCompleter The virConnectListAllNWFilters() has no extra flags yet, which simplifies things a bit. Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshNodeDeviceNameCompleter Yet again, we don't need listing by device capabilities, so flags are unused. Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshNetworkNameCompleter Reviewed-by: John Ferlan virsh: Introduce virshInterfaceNameCompleter Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshStorageVolNameCompleter This one is a bit simpler since virStoragePoolListAllVolumes() has no flags yet. Reviewed-by: John Ferlan 2018-01-24 Michal Privoznik virsh: Introduce virshStoragePoolNameCompleter Reviewed-by: John Ferlan 2018-01-22 Chen Hanxiao libvirtd: clarify the TLS conf default value setting Provide more details related to the requirement that setting one of the values requires setting all of them. Reviewed-by: John Ferlan 2018-01-22 Michal Koutný libvirtd: Explicit dependency on systemd-machined The libvirtd daemon uses systemd-machined D-Bus API when manipulating domains. The systemd-machined is D-Bus activated on demand. However, during system shutdown systemd-machined is stopped concurrently with libvirtd and virsh users also doing their final cleanup may transitively fail due to unavailability of systemd-machined. Example error message > libvirtd[1390]: 2017-12-20 18:55:56.182+0000: 32700: error : virSystemdTerminateMachine:503 : Refusing activation, D-Bus is shutting down. To circumvent this we need to explicitly specify both ordering and requirement dependency (to avoid late D-Bus activation) on systemd-machined. See [1] for the dependency debate. [1] https://lists.freedesktop.org/archives/systemd-devel/2018-January/040095.html 2018-01-22 Laine Stump qemu: auto-add generic xhci rather than NEC xhci to Q35 domains We recently added a generic XHCI USB3 controller to QEMU, and libvirt supports adding that controller rather than the NEC XHCI USB3 controller, but when auto-adding a USB controller to Q35 domains we were still adding the vendor-specific NEC controller. This patch changes to add the generic controller instead, if it's available in the QEMU binary that will be used. Reviewed-by: Pavel Hrdina 2018-01-22 Jiri Denemark qemu: Don't initialize struct utsname It breaks the build and it is not really useful for anything. 2018-01-22 Jiri Denemark qemu: Refresh caps cache after booting a different kernel Whenever a different kernel is booted, some capabilities related to KVM (such as CPUID bits) may change. We need to refresh the cache to see the changes. Reviewed-by: Daniel P. Berrange 2018-01-21 Laine Stump qemu: move qemuDomainDefValidateVideo into qemuDomainDeviceDefValidateVideo qemuDomainDefValidateVideo() (called from qemuDomainDefValidate()) is just a loop performing various checks on each video device. Rather than maintaining this separate function, just fold the validations into qemuDomainDeviceDefValidateVideo(), which is called once for each video device. 2018-01-21 Laine Stump qemu: assign correct type of PCI address for vhost-scsi when using pcie-root Commit 10c73bf1 fixed a bug that I had introduced back in commit 70249927 - if a vhost-scsi device had no manually assigned PCI address, one wouldn't be assigned automatically. There was a slight problem with the logic of the fix though - in the case of domains with pcie-root (e.g. those with a q35 machinetype), qemuDomainDeviceCalculatePCIConnectFlags() will attempt to determine if the host-side PCI device is Express or legacy by examining sysfs based on the host-side PCI address stored in hostdev->source.subsys.u.pci.addr, but that part of the union is only valid for PCI hostdevs, *not* for SCSI hostdevs. So we end up trying to read sysfs for some probably-non-existent device, which fails, and the function virPCIDeviceIsPCIExpress() returns failure (-1). By coincidence, the return value is being examined as a boolean, and since -1 is true, we still end up assigning the vhost-scsi device to an Express slot, but that is just by chance (and could fail in the case that the gibberish in the "hostside PCI address" was the address of a real device that happened to be legacy PCI). Since (according to Paolo Bonzini) vhost-scsi devices appear just like virtio-scsi devices in the guest, they should follow the same rules as virtio devices when deciding whether they should be placed in an Express or a legacy slot. That's accomplished in this patch by returning early with virtioFlags, rather than erroneously using hostdev->source.subsys.u.pci.addr. It also adds a test case for PCIe to assure it doesn't get broken in the future. 2018-01-19 Jim Fehlig nodedev: Fix failing to parse PCI address for non-PCI network devices Commit 8708ca01c added virNetDevSwitchdevFeature() to check if a network device has Switchdev capabilities. virNetDevSwitchdevFeature() attempts to retrieve the PCI device associated with the network device, ignoring non-PCI devices. It does so via the following call chain virNetDevSwitchdevFeature()->virNetDevGetPCIDevice()-> virPCIGetDeviceAddressFromSysfsLink() For non-PCI network devices (qeth, Xen vif, etc), virPCIGetDeviceAddressFromSysfsLink() will report an error when virPCIDeviceAddressParse() fails. virPCIDeviceAddressParse() also logs an error. After commit 8708ca01c there are now two errors reported for each non-PCI network device even though the errors are harmless. To avoid the errors, introduce virNetDevIsPCIDevice() and use it in virNetDevGetPCIDevice() before attempting to retrieve the associated PCI device. virNetDevIsPCIDevice() uses the 'subsystem' property of the device to determine if it is PCI. See the sysfs rules in kernel documentation for more details https://www.kernel.org/doc/html/latest/admin-guide/sysfs-rules.html 2018-01-19 Michal Privoznik Revert "qemu: monitor: do not report error on shutdown" https://bugzilla.redhat.com/show_bug.cgi?id=1536461 This reverts commit aeda1b8c56dc58b0a413acc61bbea938b40499e1. Problem is that we need mon->lastError to be set because it's used all over the place. Also, there's nothing wrong with reporting error if one occurred. I mean, if there's a thread executing an API and which currently is talking on monitor it definitely wants the error reported. 2018-01-19 Michal Privoznik Post-release version bump to 4.1.0 2018-01-19 Daniel Veillard Release of libvirt-4.0.0 * docs/news.xml: update for release * po/*.po*: regenerated 2018-01-19 Jiri Denemark qemu: Fix crash in offline migration When migrating a shutoff domain (i.e., offline migration), we have no statistics to report and thus jobInfo will be NULL in qemuMigrationFinish. Broken by me in v3.10.0-183-ge8784e7868. https://bugzilla.redhat.com/show_bug.cgi?id=1536351 Reviewed-by: Pavel Hrdina 2018-01-18 Jiri Denemark cpu: Add EPYC-IBPB CPU model This is a variant of EPYC with indirect branch prediction protection. The only difference between EPYC and EPYC-IBPB is the added "ibpb" feature. Reviewed-by: Pavel Hrdina 2018-01-18 Ján Tomko Raise the frame limit for tests After the latest CPU additions, the build fails with clang: cputest.c:905:1: error: stack frame size of 26136 bytes in function 'mymain' [-Werror,-Wframe-larger-than=] Raise the relaxed limit which is used for tests. 2018-01-18 Daniel P. Berrange qemu: avoid denial of service reading from QEMU monitor (CVE-2018-5748) We read from QEMU until seeing a \r\n pair to indicate a completed reply or event. To avoid memory denial-of-service though, we must have a size limit on amount of data we buffer. 10 MB is large enough that it ought to cope with normal QEMU replies, and small enough that we're not consuming unreasonable mem. 2018-01-17 Andrea Bolognani news: Update for 4.0.0 As usual, a bunch of changes slipped through the cracks during the development cycle. Update the release notes to include at least the most notable ones. 2018-01-17 Marc Hartmayer qemu: qemuDomainNamespaceUnlinkPaths: Return 0 in case of success Commit 7a931a4204af refactored the code and probably forgot to add this line. Reviewed-by: Boris Fiuczynski 2018-01-17 Jiri Denemark cpu: Add Skylake-Server-IBRS CPU model This is a variant of Skylake-Server with indirect branch prediction protection. The only difference between Skylake-Server and Skylake-Server-IBRS is the added "spec-ctrl" feature. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Skylake-Client-IBRS CPU model This is a variant of Skylake-Client with indirect branch prediction protection. The only difference between Skylake-Client and Skylake-Client-IBRS is the added "spec-ctrl" feature. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Broadwell-IBRS CPU model This is a variant of Broadwell with indirect branch prediction protection. The only difference between Broadwell and Broadwell-IBRS is the added "spec-ctrl" feature. The Broadwell-IBRS model in QEMU is a bit different since Broadwell got several additional features since we added it in cpu_map.xml: abm, arat, f16c, rdrand, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Broadwell-noTSX-IBRS CPU model This is a variant of Broadwell-noTSX with indirect branch prediction protection. The only difference between Broadwell-noTSX and Broadwell-noTSX-IBRS is the added "spec-ctrl" feature. The Broadwell-noTSX-IBRS model in QEMU is a bit different since Broadwell-noTSX got several additional features since we added it in cpu_map.xml: abm, arat, f16c, rdrand, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Haswell-IBRS CPU model This is a variant of Haswell with indirect branch prediction protection. The only difference between Haswell and Haswell-IBRS is the added "spec-ctrl" feature. The Haswell-IBRS model in QEMU is a bit different since Haswell got several additional features since we added it in cpu_map.xml: arat, abm, f16c, rdrand, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Haswell-noTSX-IBRS CPU model This is a variant of Haswell-noTSX with indirect branch prediction protection. The only difference between Haswell-noTSX and Haswell-noTSX-IBRS is the added "spec-ctrl" feature. The Haswell-noTSX-IBRS model in QEMU is a bit different since Haswell-noTSX got several additional features since we added it in cpu_map.xml: arat, abm, f16c, rdrand, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add IvyBridge-IBRS CPU model This is a variant of IvyBridge with indirect branch prediction protection. The only difference between IvyBridge and IvyBridge-IBRS is the added "spec-ctrl" feature. The IvyBridge-IBRS model in QEMU is a bit different since IvyBridge got several additional features since we added it in cpu_map.xml: arat, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add SandyBridge-IBRS CPU model This is a variant of SandyBridge with indirect branch prediction protection. The only difference between SandyBridge and SandyBridge-IBRS is the added "spec-ctrl" feature. The SandyBridge-IBRS model in QEMU is a bit different since SandyBridge got several additional features since we added it in cpu_map.xml: arat, vme, xsaveopt Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Westmere-IBRS CPU model This is a variant of Westmere with indirect branch prediction protection. The only difference between Westmere and Westmere-IBRS is the added "spec-ctrl" feature. The Westmere-IBRS model in QEMU is a bit different since Westmere got several additional features since we added it in cpu_map.xml: arat, pclmuldq, vme Adding them only to the -IBRS variant would confuse our CPU detection code. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cpu: Add Nehalem-IBRS CPU model This is a variant of Nehalem with indirect branch prediction protection. The only difference between Nehalem and Nehalem-IBRS is the added "spec-ctrl" feature. Thus the diff matches QEMU, but the new CPU model itself is different. The QEMU's versions of both models contain "vme" feature, while this feature is missing in libvirt's models. While we can't change the existing Nehalem CPU model, we could add "vme" to Nehalem-IBRS to make it similar to QEMU, but doing so would fool our CPU detecting code so that any Nehalem CPU with "vme" feature would be detected as Nehalem-IBRS CPU without spec-ctrl. Not adding "vme" to Nehalem-IBRS is safe as QEMU will just provide the feature anyway, which matches what happens with Nehalem (and new enough machine types). Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cputest: Add data for updated Intel(R) Core(TM) i7-5600U CPU The CPU contains the updated microcode for CVE-2017-5715. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cputest: Add data for updated AMD EPYC 7601 32-Core Processor The CPU contains the updated microcode for CVE-2017-5715. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cputest: Add data for Intel(R) Xeon(R) Gold 5115 CPU The CPU contains the updated microcode for CVE-2017-5715. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cputest: Add data for Intel(R) Xeon(R) CPU E5-2623 v4 The CPU contains the updated microcode for CVE-2017-5715. The *-guest.xml and *-json.xml CPU definitions use Skylake-Client CPU model rather than Broadwell. This is similar to Xeon-E5-2650-v4 and it is caused by our CPU model selection code when no model matches the CPU signature (family + model). We'd need to maintain a complete list of CPU signatures for our CPU models to fix this. Reviewed-by: Pavel Hrdina 2018-01-17 Jiri Denemark cputest: Add data for Intel(R) Xeon(R) CPU E5-2609 v3 The CPU contains the updated microcode for CVE-2017-5715. Reviewed-by: Pavel Hrdina 2018-01-17 Paolo Bonzini cpu: add CPU features for indirect branch prediction protection Added in QEMU commits TBD and TBD. Reviewed-by: Pavel Hrdina 2018-01-17 intrigeri On startup libvirtd runs a number of QEMU processes unconfined such as: /usr/bin/qemu-system-x86_64 -S -no-user-config -nodefaults -nographic -machine none,accel=kvm:tcg -qmp unix:/var/lib/libvirt/qemu/capabilities.monitor.sock,server,nowait -pidfile /var/lib/libvirt/qemu/capabilities.pidfile -daemonize libvirtd needs to be allowed to kill these processes, otherwise they remain running. 2018-01-17 Marc Hartmayer qemu: Fix segmentation fault when attaching a non iSCSI host device Add a check if it's a iSCSI hostdev and if it's not then don't use the union member 'iscsi'. The segmentation fault occured when accessing secinfo->type, but this can vary from case to case. Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski 2018-01-17 Daniel P. Berrange rpm: updates wrt min required fedora version Update the min fedora to 26. Use a macro to record the min versions so that the later error message is always in sync with the earlier version check. Clarify the comment that refers to guessing of dist which does not actually happen. Reviewed-by: Jiri Denemark 2018-01-17 Pavel Hrdina spec: enable bash completion only on new enough distros RHEL-6 doesn't have bash-completion package by default, it has to be installed from EPEL. Reviewed-by: Daniel P. Berrange 2018-01-16 Dan Zheng nodedev: Add the missing PCI dev checks for 'mdev_types' capability Similar to commit @f44ec9c1, commit @500cbc06 introduced a new nested 'mdev_types' capability, however the mentioned commit didn't adjust virNodeDeviceNumOfCaps and virNodeDeviceListCaps functions accordingly to provide proper support for this capability. After applying this patch the following python snippet returns the expected results: import libvirt conn = libvirt.openReadOnly('qemu:///system') devs = conn.listAllDevices() for dev in devs: if 'mdev_types' in dev.listCaps(): print dev.name(),dev.numOfCaps(),dev.listCaps() 2018-01-15 Michal Privoznik m4: Check for rl_completion_quote_character Apparently we can't assume that people run readline recent enough to have rl_completion_quote_character (added in readline-5.0 released in 2011). However, we can't compile without it. So if not present, disable readline. Reviewed-by: Andrea Bolognani 2018-01-15 Michal Privoznik tools: Always compile {virsh,virt-admin}-completer.c The functions defined in these sources are referenced all over the place, however, compiler only when building with readline. Thus when building without it linker gets sad as it can't find them. Reviewed-by: Erik Skultety 2018-01-15 Michal Privoznik vsh: Provide cmdComplete stub for readline disabled builds When building without readline, this function does nothing but return false. Without touching any of its arguments which triggers a build error. Therefore, provide a stub that has arguments marked as unused. Reviewed-by: Erik Skultety 2018-01-15 Michal Privoznik vshReadlineOptionsGenerator: Don't add already specified options to the list The current state of art is as follows: 1) vshReadlineOptionsGenerator() generate all possible --options for given command, and then 2) vshReadlineOptionsPrune() clears out already provided ones from the list. Not only this brings needless memory complexity it is also not trivial to get right. We can switch to easier approach: just don't add already specified --options in the first step. Reviewed-by: Erik Skultety 2018-01-12 Bjoern Walk tests: sysinfotest: add new test case for S390 Let's add a test case for S390 with CPU frequency information available. Test data is sampled from an IBM z13 system running kernel 4.14 on LPAR. Reviewed-by: Boris Fiuczynski 2018-01-12 Bjoern Walk util: virsysinfo: parse frequency information on S390 Let's also parse the available processor frequency information on S390 so that it can be utilized by virsh sysinfo: # virsh sysinfo ... 2964 IBM/S390 00 5000 145F07 ... Reviewed-by: Marc Hartmayer Reviewed-by: Boris Fiuczynski 2018-01-12 Andrea Bolognani travis: Skip nfs-common installation Installing nfs-common is broken on trusty since build #807 https://travis-ci.org/libvirt/libvirt/builds/326705054 It's probably a transient error on Travis' side, so just comment it out for the time being to allow builds to proceed. Reviewed-by: Daniel P. Berrange 2018-01-12 Andrea Bolognani travis: Sync packages with libvirt-jenkins-ci Make sure we install the same packages lcitool would install on the CentOS CI so that we have consistent results. The package list is current as of libvirt-jenkins-ci commit 3a559ae7bc08. Reviewed-by: Daniel P. Berrange 2018-01-12 Scott Garfinkle domcaps: Treat host models as case-insensitive strings Qemu 2.11 allows case-insensitive specification of CPU models. This patch fixes the resulting problems on (at least) POWER arch machines so that Power8 and POWER8 are not different. 2018-01-12 Jiri Denemark qemu: Fix type of a completed job Libvirt 3.7.0 and earlier libvirt reported a migration job as completed immediately after QEMU finished sending migration data at which point migration was not really complete yet. Commit v3.7.0-29-g3f2d6d829e fixed this, but caused a regression in reporting statistics for completed jobs which started reporting the job as still running. This happened because the completed job statistics including the job status are copied from the running job before we finally mark it as completed. Let's make sure QEMU_DOMAIN_JOB_STATUS_COMPLETED is always set in the completed job info even when the job has not finished yet. https://bugzilla.redhat.com/show_bug.cgi?id=1523036 Reviewed-by: Pavel Hrdina 2018-01-12 Jiri Denemark qemu: Ignore fallback CPU attribute on reconnect When reconnecting to a running domain with host-model CPU started by old libvirt which did not store the actual CPU in the status XML, we need to ignore the fallback attribute to make sure we can translate the detected host CPU model to a model which is supported by the running QEMU. https://bugzilla.redhat.com/show_bug.cgi?id=1532980 Reviewed-by: Pavel Hrdina 2018-01-12 Michal Privoznik m4: Don't enable bash-completion by default Due to the way that check logic was written we basically enabled bash completion whenever readline was enabled. This is not right because it made bash-completion pkg-config module required. Reviewed-by: Pavel Hrdina 2018-01-11 Michal Privoznik virt-admin: Introduce vshAdmServerCompleter virsh: Introduce virshDomainInterfaceCompleter For given domain fetch list of defined interfaces. This can be used for commands like domif-getlink and others. If available, the interface name is returned (e.g. "vnet0", usually available only for running domains), if not the MAC address is returned. Moreover, the detach-interface command requires only MAC address and therefore we have new flag that forces the completer to return just the MAC address. virsh: Introduce virshDomainNameCompleter Now that we have everything prepared let the fun begin. This completer is very simple and returns domain names. Moreover, depending on the command it can return just a subset of domains (e.g. only running/paused/transient/.. ones). 2018-01-11 Michal Privoznik tools: Provide bash autompletion file The only purpose of this file is to be sourced. After that one can use completion even for their bash: # virsh list -- --all --inactive ... 2018-01-11 Michal Privoznik vsh: Introduce complete command This command is going to be called from bash completion script in the following form: virsh complete -- start --domain Its only purpose is to return list of possible strings for completion. Note that this is a 'hidden', unlisted command and therefore there's no documentation to it. 2018-01-11 Michal Privoznik vsh: Filter --options Similarly to the previous commit, once we've presented an --option for a command to the user it makes no sense to offer it again. Therefore, we can prune all already specified options. For instance, after this patch: virsh # migrate --verbose will no longer offer --verbose option. 2018-01-11 Michal Privoznik vsh: Prune string list returned by completer Instead of having completers prune returned string list based on user's input we can do that right after the callback is called. Only strings matching the prefix will be presented to the user then. vsh: Call vshCmdOptDef completer Now that we have everything prepared we can call options' completer again. At the same time, pass partially parsed input to the completer callback - it will help the callbacks to narrow down the list of returned options based on user's input. For instance, if the completer is supposed to return list of interfaces depending on user input it may return just those interfaces defined for already specified domain. Of course, completers might ignore this parameter. vsh: Fix vshCompleter signature The first argument passed to this function is vshControl *. There's no need to use void pointer. util: Introduce virStringListMerge For two string lists merge one into the other one. 2018-01-11 Michal Privoznik vshCommandOpt: Allow caller avoiding assert() In the future, completer callbacks will receive partially parsed command (and thus possibly incomplete). However, we still want them to use command options fetching APIs we already have (e.g. vshCommandOpt*()) and at the same time don't report any errors (nor call any asserts). Reviewed-by: John Ferlan 2018-01-11 Michal Privoznik vshReadlineParse: Use string list It's better to fetch list of either commands or options just once and then iterate over it. Moreover, it makes future completers way simpler as they will return string lists too. vshReadlineParse: Escape returned results if needed When returning a string that needs escaping there are two scenarios that can happen. Firstly, user already started the string with a quote (or double quote) in which case we don't need to do anything - readline takes care of that. However, if they haven't typed anything yet, we need to escape the string ourselves. 2018-01-11 Michal Privoznik vshReadlineParse: Drop code duplication Now that we have a way of retrieving partly parsed command we don't need duplicate code that parses the user's input. Yes, this code removes call of opt's completer, but: a) current implementation is broken anyway, and b) it will be added back shortly 2018-01-11 Michal Privoznik vshCommandStringParse: Allow retrieving partial result In the future, this function is going to be called from vshReadlineParse() to provide parsed input for completer callbacks. The idea is to allow the callbacks to provide more specific data. For instance, for the following input: virsh # domifaddr --domain fedora --interface the --interface completer callback is going to be called. Now, it is more user friendly if the completer offers only those interfaces found in 'fedora' domain. But in order to do that it needs to be able to retrieve partially parsed result. 2018-01-11 Michal Privoznik vshCommandParse: Don't leak @tkdata When parsing cmd line which has "--" on it, this is leaked. Problem is, parser->getNextArg() allocates new string and stores it into tkdata. But as soon as "--" is detected 'continue' is issued without any free of the allocated memory. ==5304== 3 bytes in 1 blocks are definitely lost in loss record 1 of 782 ==5304== at 0x4C2AF50: malloc (vg_replace_malloc.c:299) ==5304== by 0x8BB5AA9: strdup (strdup.c:42) ==5304== by 0x55842CA: virStrdup (virstring.c:941) ==5304== by 0x172B21: _vshStrdup (vsh.c:162) ==5304== by 0x175E8E: vshCommandArgvGetArg (vsh.c:1622) ==5304== by 0x17551D: vshCommandParse (vsh.c:1418) ==5304== by 0x175F25: vshCommandArgvParse (vsh.c:1638) ==5304== by 0x130940: virshParseArgv (virsh.c:820) ==5304== by 0x130C49: main (virsh.c:922) 2018-01-11 Michal Privoznik vsh: Drop useless check for cmd != NULL All our internal *Free() functions are capable of handling NULL. vsh: Drop useless check for opts != NULL All our internal *Free() functions are capable of handling NULL. 2018-01-11 John Ferlan docs: Add missing encryption type Update the text to include "luks" as a possible value. 2018-01-11 Andrea Bolognani tests: Break symlink loop distcheck, and possibly more stuff, breaks because of it. 2018-01-10 Bjoern Walk tests: virhostcputest: testcase for S390 system Let's add a testcase for a S390 system running kernel version 4.14 on LPAR. Reviewed-by: Marc Hartmayer Reviewed-by: Boris Fiuczynski 2018-01-10 Bjoern Walk util: virhostcpu: parse frequency information on S390 Since kernel version 4.7, processor frequency information is available on S390. Let's adjust the parser so this information shows up for virsh nodeinfo: # virsh nodeinfo CPU model: s390x CPU(s): 8 CPU frequency: 5000 MHz CPU socket(s): 1 Core(s) per socket: 8 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 16273908 KiB Reviewed-by: Marc Hartmayer Reviewed-by: Boris Fiuczynski 2018-01-10 John Ferlan storage: Complete implementation volume by hash object Alter the volume logic to use the hash tables instead of forward linked lists. There are three hash tables to allow for fast lookup by name, target.path, and key. Modify the virStoragePoolObjAddVol to place the object in all 3 tables if possible using self locking RWLock on the volumes object. Conversely when removing the volume, it's a removal of the object from the various hash tables. Implement functions to handle remote ForEach and Search Volume type helpers. These are used by the disk backend in order to facilitate adding a primary, extended, or logical partition. Implement the various VolDefFindBy* helpers as simple (and fast) hash lookups. The NumOfVolumes, GetNames, and ListExport helpers are all implemented using standard for each hash table calls. 2018-01-10 John Ferlan storage: Introduce _virStorageVolObj[List] Prepare for hash table volume lists by creating the object infrastructure for a Volume Object and Volume Object List The _virStorageVolObj will contain just a pointer to the "current" (and live) volume definition. The _virStorageVolObjList will contain three hash tables, one for each of the lookup options allowed for a volume. 2018-01-10 John Ferlan storage: Modify virStorageBackendDiskMakeDataVol logic Alter the logic such that we only add the volume to the pool once we've filled in all the information and cause failure to go to a common error: label. Patches to place the @vol into a few hash tables will soon "require" that at least the keys (name, target.path, and key) be populated with valid data. storage: When delete volume avoid disk backend removal For a disk backend, the deleteVol code will clear all the volumes in the pool and perform a pool refresh, thus the storageVolDeleteInternal should not use access @voldef after deleteVol succeeds. 2018-01-10 Jiri Denemark cpu_x86: Copy CPU signature from ancestor When specifying a new CPU model in cpu_map.xml as an extension to an existing model, we forgot to copy the signature (family + model) from the original CPU model. We don't use this way of specifying CPU models, but it's still supported and it becomes useful when someone wants to quickly hack up a CPU model for testing or when creating additional variants of existing models to help with fixing some spectral issues. Reviewed-by: Pavel Hrdina 2018-01-10 Jiri Denemark cpu_x86: Add debug messages to x86DecodeUseCandidate When translating CPUID data into CPU model + features, the code sometimes uses an unexpected CPU model. There may be several reasons for this, starting with wrong expectations and ending with an actual bug in our code. These debug messages will help determining the reason. Reviewed-by: Pavel Hrdina 2018-01-10 Jiri Denemark cputest: Fix cpu-cpuid.py diff command The cpuidMap in cpu-cpuid.py was created for converting old data files (with QEMU's feature-words bits) to the new model-expansion based data. When I added tests for CPU live update based on disabled/enabled feature lists I shamelessly used the existing cpuidMap for generating the *-{enabled,disabled}.xml data files. Thus any new CPUID bits which are not present in the original cpuidMap would be ignored. The correct thing to do is to use cpu_map.xml. All data files were fixed by running the following command: ./cpu-cpuid.py diff *.json Reviewed-by: Pavel Hrdina 2018-01-09 Christian Ehrhardt apparmor, virt-aa-helper: drop static channel rule This is now covered by DomainSetPathLabel being implemented in apparmor. 2018-01-09 Christian Ehrhardt security, apparmor: add (Set|Restore)ChardevLabel Since 1b4f66e "security: introduce virSecurityManager (Set|Restore)ChardevLabel" this is a public API of security manager. Implementing this in apparmor avoids miss any rules that should be added for devices labeled via these calls. 2018-01-09 Christian Ehrhardt security: full path option for DomainSetPathLabel virSecurityManagerDomainSetPathLabel is used to make a path known to the security modules, but today is used interchangably for - paths to files/dirs to be accessed directly - paths to a dir, but the access will actually be to files therein Depending on the security module it is important to know which of these types it will be. The argument allowSubtree augments the call to the implementations of DomainSetPathLabel that can - per security module - decide if extra actions shall be taken. For now dac/selinux handle this as before, but apparmor will make use of it to add a wildcard to the path that was passed. 2018-01-09 Christian Ehrhardt security, apparmor: implement domainSetPathLabel This came up in discussions around huge pages, but it will cover more per guest paths that should be added to the guests apparmor profile: - keys via qemuDomainWriteMasterKeyFile - per domain dirs via qemuProcessMakeDir - memory backing paths via qemuProcessBuildDestroyMemoryPathsImpl 2018-01-09 Jamie Strandboge apparmor, virt-aa-helper: Allow access to /sys/bus/usb/devices Required to generate correct profiles when using usb passthrough. Bug-Ubuntu: https://bugs.launchpad.net/bugs/565691 Acked-by: Jamie Strandboge Acked-by: Intrigeri 2018-01-09 Jamie Strandboge apparmor, libvirtd: Allow ixr to /var/lib/libvirt/virtd* This is required for the ebtables functionality added in libvirt 0.8.0. 2018-01-09 Christian Ehrhardt apparmor, libvirt-qemu: qemu won't call qemu-nbd While libvirtd might do so, qemu itself as a guest will not need to call qemu-nbd so remove it from the profile. 2018-01-09 Christian Ehrhardt apparmor, libvirt-qemu: add default pki path of libvirt-spice Adding the PKI path that is used as default suggestion in src/qemu/qemu.conf If people use non-default paths they should use local overrides but the suggested defaults we should open up. This is the default path as referenced by src/qemu/qemu.conf in libvirt. While doing so merge the several places we have to cover PKI access into one. Bug-Ubuntu: https://bugs.launchpad.net/bugs/1690140 2018-01-09 Jamie Strandboge apparmor, libvirt-qemu: Allow qemu-block-extra libraries Allows (multi-arch enabled) access to libraries under the /usr/lib/@{multiarch}/qemu/*.so path in the Debian/Ubuntu qemu-block-extra package and all such libs for the paths of rpm qemu-block-* packages. Bug-Ubuntu: https://bugs.launchpad.net/bugs/1554761 2018-01-09 Stefan Bader apparmor, libvirt-qemu: Silence lttng related deny messages Prevent denial messages related to attempted reads on lttng files from spamming the logs. Bug-Ubuntu: https://bugs.launchpad.net/bugs/1432644 2018-01-09 Serge Hallyn apparmor, libvirt-qemu: Allow use of sgabios Bug-Ubuntu: https://bugs.launchpad.net/bugs/1393548 2018-01-09 Michal Privoznik qemu: Prepare BIOS/UEFI when starting a domain https://bugzilla.redhat.com/show_bug.cgi?id=1527740 Users might use a block device as UEFI VAR store. Or even have OVMF stored there. Therefore, when starting a domain and separate mount namespace is used, we have to create all the /dev entries that are configured for the domain. Reviewed-by: John Ferlan 2018-01-08 Chen Hanxiao news: add change of hot unplug redirdev Reviewed-by: John Ferlan 2018-01-08 Chen Hanxiao qemu: Add support for hot unplugging redirdev device Commit id '162efa1a' added support hotplug a redirdev, but did not add the hot unplug. This patch will add that support to allow usage of the detach-device --live on the device. Reviewed-by: John Ferlan 2018-01-08 Andrea Bolognani util: Don't report CPU frequency for ARM hosts Some ARM platforms, such as the original Raspberry Pi, report the CPU frequency in the BogoMIPS field of /proc/cpuinfo, so libvirt parsed that field and returned it through its API. However, not only many more boards don't report any value there, but several - including ARMv8-based server hardware, and even the more recent Raspberry Pi 3 - use this field as originally intended: to report the BogoMIPS value instead of the CPU frequency. Since we have no way of detecting how the field is being used, it's better to report no information at all rather than something ludicrous like "your shiny 96-core aarch64 virtualization host's CPUs are running at a whopping 100 MHz". Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1206353 Reviewed-by: John Ferlan 2018-01-08 Andrea Bolognani util: Improve CPU frequency parsing Make the parser both more strict, by not ignoring errors reported by virStrToLong_ui(), and more permissive, by not failing due to unrelated fields which just happen to have a know prefix and accepting any amount of whitespace before the numeric value. Reviewed-by: Bjoern Walk 2018-01-08 Andrea Bolognani util: Print architecture name in /proc/cpuinfo parser Instead of a generic "your architecture", print the actual architecture name. Reviewed-by: Bjoern Walk 2018-01-08 Bjoern Walk util: virhostcpu: factor out frequency parsing All different architectures use the same copy-pasted code to parse processor frequency information from /proc/cpuinfo. Let's extract that code into a function to avoid repetition. We now also tolerate if the parsing of /proc/cpuinfo is not successful and just report a warning instead of bailing out and abandoning the rest of the CPU information. Reviewed-by: Marc Hartmayer Reviewed-by: Boris Fiuczynski Reviewed-by: Andrea Bolognani Reviewed-by: John Ferlan 2018-01-08 Andrea Bolognani tests: Add host CPU data for Moonshot (RHEL 7.4) Reviewed-by: John Ferlan 2018-01-08 Michal Privoznik qemuDomainAttachDeviceMknodHelper: Remove symlink before creating it https://bugzilla.redhat.com/show_bug.cgi?id=1528502 So imagine you have /dev/blah symlink which points to /dev/sda. You attach /dev/blah as disk to your domain. Libvirt correctly creates the /dev/blah -> /dev/sda symlink in the qemu namespace. However, then you detach the disk, change the symlink so that it points to /dev/sdb and tries to attach the disk again. This time, however, the attach fails (well, qemu attaches wrong disk) because the code assumes that symlinks don't change. Well they do. This is inspired by test fix written by Eduardo Habkost. Reviewed-by: Andrea Bolognani 2018-01-05 Shivaprasad G Bhat qemu: Add support for pseries machine's max-cpu-compat= parameter When the -machine pseries,max-cpu-compat=X is supported use machine parameter instead of -cpu host,compat=X parameter as that is deprecated now with qemu >= v2.10. Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1519146 Reviewed-by: Andrea Bolognani 2018-01-05 Shivaprasad G Bhat qemu: Add capability for pseries machine's max-cpu-compat= parameter Reviewed-by: Andrea Bolognani 2018-01-05 Michal Privoznik qemuDomainDiskChangeSupported: Forbid alias change Since we have user aliases it may happen that users want to change it using 'update-device'. Instead of ignoring it silently, error out loudly. Note that we don't limit the check just for "ua-" prefixes because users might try to change libvirt generated aliases too. Reviewed-by: John Ferlan 2018-01-04 John Ferlan qemu: Alter dump-guest-memory command generation The qemuMonitorJSONMakeCommand can properly handle a NULL string by using the "S:" parameter instead of "s:", so let's use that of having in if/else condition that only adds the "s:". qemu: Clean up style for the qemuDumpToFd definition Alter the function definition to follow more recent style test: Use virStoragePoolObjIsDuplicate for storage define/create Avoid the chance that there could be a duplicate storage pool UUID or Name from the test driver storage pool define/create functions. conf: Use bool for @check_active parameter Use a bool as that's how the variable is used in the function. conf: Need to unlock pools on object allocation failure The RW pool could be left locked if allocation fails. 2018-01-04 Paolo Bonzini qemu: capabilities: force update if the microcode version does not match A microcode update can cause the CPUID bits to change; an example from the past was the update that disabled TSX on several Haswell and Broadwell machines. Therefore, place microcode version in the virQEMUCaps struct and XML, and rebuild the cache if the versions do not match. 2018-01-04 Paolo Bonzini conf: include x86 microcode version in virsh capabilities A microcode update can cause the CPUID bits to change; an example from the past was the update that disabled TSX on several Haswell and Broadwell machines. In order to track the x86 microcode version in the QEMU capabilities, we have to fetch it and store it in the host CPU. This also makes the version visible in "virsh capabilities", which is a nice side effect. 2018-01-04 Jiri Denemark cpu_x86: Rename virCPUx86MapInitialize The function will be used to initialize internal data of the x86 CPU driver (including the CPU map). 2018-01-04 Paolo Bonzini util: introduce virHostCPUGetMicrocodeVersion This new API reads host's CPU microcode version from /proc/cpuinfo. Unfortunately, there is no other way of reading microcode version which would be usable from both system and session daemon. 2018-01-04 Paolo Bonzini util: add virFileReadHeaderQuiet wrapper around virFileReadHeaderFD 2018-01-04 John Ferlan conf: Fix generating addresses for SCSI hostdev https://bugzilla.redhat.com/show_bug.cgi?id=1519130 Commit id 'dc692438' reverted the automagic addition of a SCSI controller attempt during virDomainHostdevAssignAddress; however, the logic to determine where to place the next_unit depended upon the "new" controller being added. Without the new controller the the next time through the call for the next SCSI hostdev found would result in the "next_unit" never changing from 0 (zero) and as a result the addition of the device will fail due to being a duplicate unit number of the first with the error message: virDomainDefCheckDuplicateDriveAddresses:$line : unsupported configuration: SCSI host address controller='0' bus='1' target='0' unit='0' in use by another SCSI host device So instead of walking the controller list looking for SCSI controllers, all we can do is "pretend" that they exist and allow other code to create them later as necessary. 2018-01-04 John Ferlan conf: Use existing SCSI hostdev model to create new In virDomainDefMaybeAddHostdevSCSIcontroller when we add a new controller because someone neglected to add one or we're adding one because the existing one is full, we should copy over the model number from the existing controller since whatever we create should at least have the same characteristics as the one we cannot use because it's full. NB: This affects the existing hostdev-scsi-autogen-address test which would add a default ('lsi') SCSI controller for the various scsi_host's that would create a controller for the hostdev. 2018-01-04 John Ferlan qemu: Use same model when adding hostdev SCSI controller When qemuDomainFindOrCreateSCSIDiskController adds a controller, let's use the same model as a currently found controller under the assumption that the reason to add the controller in hotplug is because virDomainHostdevAssignAddress determined that there were too many devices on the existing controller, but only assigned a new controller index and did not add a new controller and we desire to use the same controller model as any existing controller and not take a chance that qemuDomainSetSCSIControllerModel would use a default that may be incompatible. 2018-01-04 Pavel Hrdina rpc: remove redundant logic Introduced by commit <0eaa59dce1>. That comparison already returns true or false. 2018-01-04 John Ferlan nodedev: Move device enumumeration out of nodeStateInitialize Let's move the udevEnumerateDevices into a thread to "speed up" the initialization process. If the enumeration fails we can set the Quit flag to ensure that udevEventHandleCallback will not run. Reviewed-by: Erik Skultety 2018-01-04 Marc Hartmayer rpc: Replace virNetServerClientNeedAuth with virNetServerClientIsAuthenticated Replace virNetServerClientNeedAuth with virNetServerClientIsAuthenticated because it makes it clearer what it means. Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann 2018-01-04 Marc Hartmayer rpc: Remove virNetServerClientNeedAuthLocked 'Squash' virNetServerClientNeedAuthLocked into virNetServerClientNeedAuth and remove virNetServerClientNeedAuthLocked as it's not longer needed. Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann 2018-01-04 Marc Hartmayer tests: virnetdaemontest: Enable testing for 'auth_pending' Enable testing for 'auth_pending' in the virnetdaemon test case. Reviewed-by: Boris Fiuczynski Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: virnetserver: Fix race on srv->nclients_unauth There is a race between virNetServerProcessClients (main thread) and remoteDispatchAuthList/remoteDispatchAuthPolkit/remoteSASLFinish (worker thread) that can lead to decrementing srv->nclients_unauth when it's zero. Since virNetServerCheckLimits relies on the value srv->nclients_unauth the underrun causes libvirtd to stop accepting new connections forever. Example race scenario (assuming libvirtd is using policykit and the client is privileged): 1. The client calls the RPC remoteDispatchAuthList => remoteDispatchAuthList is executed on a worker thread (Thread T1). We're assuming now the execution stops for some time before the line 'virNetServerClientSetAuth(client, 0)' 2. The client closes the connection irregularly. This causes the event loop to wake up and virNetServerProcessClient to be called (on the main thread T0). During the virNetServerProcessClients the srv lock is hold. The condition virNetServerClientNeedAuth(client) will be checked and as the authentication is not finished right now virNetServerTrackCompletedAuthLocked(srv) will be called => --srv->nclients_unauth => 0 3. The Thread T1 continues, marks the client as authenticated, and calls virNetServerTrackCompletedAuthLocked(srv) => --srv->nclients_unauth => --0 => wrap around as nclient_unauth is unsigned 4. virNetServerCheckLimits(srv) will disable the services forever To fix it, add an auth_pending field to the client struct so that it is now possible to determine if the authentication process has already been handled for this client. Setting the authentication method to none for the client in virNetServerProcessClients is not a proper way to indicate that the counter has been decremented, as this would imply that the client is authenticated. Additionally, adjust the existing test cases for this new field. Reviewed-by: Boris Fiuczynski 2018-01-04 Marc Hartmayer rpc: Introduce virNetServerSetClientAuthenticated Combine virNetServerClientSetAuth(client, VIR_NET_SERVER_SERVICE_AUTH_NONE) and virNetServerTrackCompletedAuth into one new function named virNetServerSetClientAuthenticated. After using this new function the function virNetServerTrackCompletedAuth was superfluous and is therefore removed. In addition, it is not very common that a '{{function}}' (virNetServerTrackCompletedAuth) does more than just the locking compared to '{{function}}Locked' (virNetServerTrackCompletedAuthLocked). virNetServerTrackPendingAuth was already superfluous and therefore it's also removed. Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: Correct locking and simplify the function The lock for @client must not only be held for the duration of checking whether the client wants to close, but also for as long as we're closing the client. The same applies to the tracking of authentications. 2018-01-04 Marc Hartmayer rpc: Refactor the condition whether a client needs authentication Add virNetServerClientAuthMethodImpliesAuthenticated() for deciding whether a authentication method implies that a client is automatically authenticated or not. Use this new function in virNetServerClientNeedAuthLocked(). Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann 2018-01-04 Marc Hartmayer rpc: First test if authentication is required This makes the code more efficient. Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: Be more precise in which cases the authentication is needed and introduce *Locked Be more precise in which cases the authentication is needed and introduce *Locked. Reviewed-by: Boris Fiuczynski Reviewed-by: Bjoern Walk Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: Add typedef for the anonymous enum used for authentication methods Add typedef for the anonymous enum used for the authentication methods and remove the default case. This allows the usage of the type in a switch statement and taking advantage of the compilers feature to detect uncovered cases. Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: Use the enum value instead of a numerical value Reviewed-by: Boris Fiuczynski Reviewed-by: Bjoern Walk Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer tests: virnetserverclienttest: Fix memory leak @client Direct leak of 104 byte(s) in 1 object(s) allocated from: #0 0x7f904bfbe12b (/lib64/liblsan.so.0+0xe12b) #1 0x7f904ba0ad67 in virAlloc ../../src/util/viralloc.c:144 #2 0x7f904bbc11a4 in virNetMessageNew ../../src/rpc/virnetmessage.c:42 #3 0x7f904bbb8e77 in virNetServerClientNewInternal ../../src/rpc/virnetserverclient.c:392 #4 0x7f904bbb9921 in virNetServerClientNew ../../src/rpc/virnetserverclient.c:440 #5 0x402ce5 in testIdentity ../../tests/virnetserverclienttest.c:55 #6 0x403bed in virTestRun ../../tests/testutils.c:180 #7 0x402c1e in mymain ../../tests/virnetserverclienttest.c:146 #8 0x404c80 in virTestMain ../../tests/testutils.c:1119 #9 0x4030d5 in main ../../tests/virnetserverclienttest.c:152 #10 0x7f9047f7f889 in __libc_start_main (/lib64/libc.so.6+0x20889) Indirect leak of 4 byte(s) in 1 object(s) allocated from: #0 0x7f904bfbe12b (/lib64/liblsan.so.0+0xe12b) #1 0x7f904ba0adc7 in virAllocN ../../src/util/viralloc.c:191 #2 0x7f904bbb8ec7 in virNetServerClientNewInternal ../../src/rpc/virnetserverclient.c:395 #3 0x7f904bbb9921 in virNetServerClientNew ../../src/rpc/virnetserverclient.c:440 #4 0x402ce5 in testIdentity ../../tests/virnetserverclienttest.c:55 #5 0x403bed in virTestRun ../../tests/testutils.c:180 #6 0x402c1e in mymain ../../tests/virnetserverclienttest.c:146 #7 0x404c80 in virTestMain ../../tests/testutils.c:1119 #8 0x4030d5 in main ../../tests/virnetserverclienttest.c:152 #9 0x7f9047f7f889 in __libc_start_main (/lib64/libc.so.6+0x20889) SUMMARY: LeakSanitizer: 108 byte(s) leaked in 2 allocation(s). Reviewed-by: John Ferlan 2018-01-04 Marc Hartmayer rpc: Remove duplicate declaration of virNetServerAddClient Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski Reviewed-by: Stefan Zimmermann Reviewed-by: John Ferlan 2018-01-04 Eduardo Habkost qemu_cgroup: Fix 'rc' argument on virDomainAuditCgroupPath() calls All calls to virDomainAuditCgroupPath() were passing 'rc == 0' as argument, when it was supposed to pass the 'rc' value directly. As a consequence, the audit events that were supposed to be logged (actual cgroup changes) were never being logged, and bogus audit events were logged when using regular files as disk image. Fix all calls to use the return value of virCgroup{Allow,Deny}Device*() directly as the 'rc' argument. 2018-01-04 Michal Privoznik virstringtest: Fix alignment of backslashes We don't try to right align the backslashes anymore. 2018-01-03 Eric Blake maint: update to latest gnulib This pulls in updates for the 2018 copyright year, among other things. 2018-01-03 Cédric Bosdonnat apparmor: fix virt-aa-helper profile Fix rule introduced by commit 0f33025a: * to handle /var/run not being a symlink to /run * to be properly parsed: missing comma at the end. 2018-01-03 Julio Faracco storage: Fixing missing 'backingStore' tag from volume XML dumps. After commit a693fdb 'vol-dumpxml' missed the ability to show backingStore information. This commit adds a volume type for files that fixes this problem. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1529663 2018-01-03 Michal Privoznik qemuBuildMemPathStr: Forbid memoryBacking/access for non-numa case https://bugzilla.redhat.com/show_bug.cgi?id=1448149 If a domain has no numa nodes, that means we don't put any memory-backend-file onto the qemu command line. That in turn means we can't set access='shared'. Therefore, we should produce an error instead of ignoring the setting silently. 2018-01-03 Peter Krempa qemu: monitor: Decrease logging verbosity The PROBE macro used in qemuMonitorIOProcess and the VIR_DEBUG message in qemuMonitorJSONIOProcess create a lot of logging churn when debug logging is enabled during monitor communication. The messages logged from the PROBE macro are rather useless since they are reporting the partial state of receiving the reply from qemu. The actual full reply is still logged in qemuMonitorJSONIOProcessLine once the full message is received. 2018-01-03 Peter Krempa util: probe: Add quiet versions of the "PROBE" macro PROBE macro adds a logging entry, when used in places seeing a lot of traffic this can cause a significant slowdown. 2018-01-03 Chen Hanxiao util: virstring: Tweak a few more descriptions. There are a few more description-related issues that commit @9026d115 forgot to address. 2018-01-03 Cédric Bosdonnat lxc: set a hostname based on the container name Set a transient hostname on containers. The hostname is computed from the container name, only keeping the valid characters [a-zA-Z0-9-] in it. This filtering is based on RFC 1123 and allows a digit to start the hostname. Add virStringFilterChars() string utility Add a function to filter a string based on a list of valid characters. 2018-01-03 Chen Hanxiao util: fix the description of virStringSearch There's no argument named @result, use @matches instead. 2018-01-03 Christian Ehrhardt apparmor: allow unix stream for p2p migrations On live migration with --p2p like: $ virsh migrate --live --p2p kvmguest-bionic-normal \ qemu+ssh://10.6.221.80/system We hit an apparmor deny like: apparmor="DENIED" operation="file_inherit" profile="/usr/sbin/libvirtd" pid=23477 comm="ssh" family="unix" sock_type="stream" protocol=0 requested_mask="send receive" denied_mask="send" addr=none peer_addr=none peer="unconfined" The rule is not perfect, but can't be restricted further at the moment (new upstream kernel features needed). For now the lack of a profile on the peer as well as comm not being a conditional on rules do not allow to filter further. 2018-01-03 Michal Privoznik src: Export virDomainMemoryAccessType*String These are already exported at header file level because of VIR_ENUM_DECL being in numa_conf.h. However, they are not being exported at object level because of missing libvirt_private.syms record. Reviewed-by: John Ferlan 2018-01-02 Andrea Bolognani docs: Fix serial console configuration examples 2018-01-02 Andrea Bolognani qemu: Enforce vCPU hotplug granularity constraints QEMU 2.7 and newer don't allow guests to start unless the initial vCPUs count is a multiple of the vCPU hotplug granularity, so validate it and report an error if needed. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1283700 2018-01-02 Andrea Bolognani qemu: Invert condition nesting in qemuDomainDefValidate() While at the moment we're only performing a single check that is connected to vCPU hotplugging, we're going to introduce a second one soon. Move the topology check underneath the capability check to make that easier; since, after this change, the 'topologycpus' variable doesn't need to have function scope, we move its declaration to the inner scope as well. The comments around the check are modified in order to explain the different QEMU versions involved. 2018-01-02 Pavel Hrdina conf: honor maxnames in nodeListDevices API Introduced by commit <4ae9dbea99c>. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1528572 2017-12-21 John Ferlan qemu: Move TCP and haveTLS checks into qemuDomainDelChardevTLSObjects Similar to qemuDomainAddChardevTLSObjects let's move the chardev source must be TCP and it has the @haveTLS flag set checks before trying to delete the TLS objects. For the Chr device this represents no change; however, for RNG device this is an additionaly check that was missed in commit id '68808516'. Before adding the objects, TCP and haveTLS are checked. 2017-12-21 John Ferlan qemu: Introduce qemuDomainDelChardevTLSObjects Let's make a comment deletion helper similar to the Add helper that can be called after the ExitMonitor. The modify qemuDomainRemoveChrDevice and qemuDomainRemoveRNGDevice to call the helper instead of inlining the copy and pasted code. 2017-12-21 Cédric Bosdonnat netserver: close clients before stopping all drivers So far clients were closed when disposing the daemon, after the state driver cleanup. This was leading to libvirtd crashing at shutdown due to missing driver. Moving the client close in virNetServerClose() fixes the problem. Reviewed-by: Erik Skultety 2017-12-20 Cédric Bosdonnat virt-aa-helper: handle more disk images virt-aa-helper needs read access to the disk image to resolve symlinks and add the proper rules to the profile. Its profile whitelists a few common paths, but users can place their images anywhere. This commit helps users allowing access to their images by adding their own rules in apparmor.d/local/usr.lib.libvirt.virt-aa-helper. This commit also adds rules to allow reading files named: - *.raw as this is a rather common disk image extension - /run/libvirt/**[vd]d[a-z] as these are used by virt-sandbox 2017-12-18 Daniel P. Berrange autogen.sh: tell user the correct make command When autogen.sh finishes it helpfully prints "Now type 'make' to compile libvirt." which is fine if on a host with GNU make, but on *BSD running 'make' will end in tears. We should tell users to run 'gmake' on these platforms. If 'gmake' doesn't exist then we should report an error too "GNU make is required to build libvirt" Reviewed-by: Andrea Bolognani 2017-12-18 Jim Fehlig Improve filtering of Xen domain0 in libvirt-guests The list_guests function in libvirt-guests uses 'grep -v' to filter Xen domain0 from a list of guests. If domain0 is the only item in the list, 'grep -v' returns 1, causing the 'stop' operation to fail when action is 'suspend'. Improve the filtering by using sed to remove domain0 from the list of guests. 2017-12-18 Erik Skultety admin: Use the connection to determine a client is connected readonly Prior to this change, we relied solely on the inherited readonly attribute of a service's socket. This only worked for our UNIX sockets (and only to some degree), but doesn't work for TCP sockets which are RW by default, but such connections support RO as well. This patch forces an update on the client object once we have established a connection to reflect the nature of the connection itself rather than relying on the underlying socket's attributes. Clients connected to the admin server have always been connected as RW only. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1524399 2017-12-15 Jim Fehlig libxl: mark domain0 as persistent A Xen domain0 is better described as a persistent domain. Mark it as such during intialization. 2017-12-15 Andrea Bolognani cfg.mk: Simplify backslash alignment check The use of [[:blank:]] was intended to cover tabs as well, but it couldn't possibly work in its current form, so the regex was tweaked in d09429abe826. With the original reason for using [[:blank:]] now gone, we can replace its usage with plain spaces. A comment about the purpose of the check is added as well. 2017-12-14 Laine Stump qemu: log error on attempts to set filterref on an OVS-connected interface ebtables/iptables processing is skipped for any interface connected to Open vSwitch (they have their own packet filtering), likewise for midonet (according to http://blog.midokura.com/2016/04/midonet-rule-chains), but libvirt would allow adding a to interfaces connected in these ways, so the user might mistakenly believe they were being protected. This patch checks for a non-NULL element for an interface (or its network) and logs an error if and are both present. This could cause some previously working domains to no longer start, but that's really the whole point of this patch - to warn people that their filterref isn't protecting them as they might have thought. I don't bother checking this during post-parse validation, because such a check would be incomplete - it's possible that a network would have a that would be applied to an interface, and you can't know that until the domain is started. Resolves: https://bugzilla.redhat.com/1502754 2017-12-14 Laine Stump qemu: delete exist bandwidth restrictions when they are removed from config When the of an interface is changed with update-device, the old settings are cleared with tc, then new settings added with tc. But if the qemu: remove input device after receiving the event Also call qemuDomainRemoveInputDevice if we receive the event after the Detach API ends. Commit 67486bb failed to include this. https://bugzilla.redhat.com/show_bug.cgi?id=1524837 Reviewed-by: Erik Skultety 2017-12-14 Peter Krempa qemu: domain: Parse and format relPath into disk source private data Register the helpers directly to format and parse the data. https://bugzilla.redhat.com/show_bug.cgi?id=1523261 2017-12-14 Peter Krempa util: storage: Add helpers to parse and format relPath into privateData This will be the first private piece of data that will need to be stored in the XML for some drivers. Add helpers which will do it. 2017-12-14 Peter Krempa conf: Add infrastructure for disk source private data XML VM drivers may need to store additional private data to the status XML so that it can be restored after libvirtd restart. Since not everything is needed add a callback infrastructure, where VM drivers can add only stuff they need. Note that the private data is formatted as a sub-element of the or sub-element. This is done since storing it out of band (in the VM private data) would require a complex matching process to allow to put the data into correct place. 2017-12-14 John Ferlan docs: Fix news.xml syntax Commit id '1241e487' used <interface> which is not valid syntax for a . 2017-12-13 Jim Fehlig news: mention multiple IP addresses support for Xen 2017-12-13 John Ferlan util: Report error if vhost-scsi device file cannot be found https://bugzilla.redhat.com/show_bug.cgi?id=1523564 If the vhost-scsi device file cannot be found, the generic error "error: An error occurred, but the cause is unknown" is returned. Let's add a real error message to make it clear why the failure occurred. 2017-12-13 John Ferlan util: Fix error path in virSCSIVHostOpenVhostSCSI We cannot be sure someone initialized the passed *vhostfd and we certainly don't want or need to be calling VIR_FORCE_CLOSE on what probably is -1. So let's just return -1 immediately. 2017-12-13 John Ferlan qemu: Need to assign PCI address to vhost-scsi Commit id '70249927b' neglected to cover this case because the test had taken the "shortcut" to already add the
; however, when the PCI address assignment code was adjusted by commit id '70249927' the vhost-scsi (VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_SCSI_HOST) wasn't covered thus returning a 0 for pciFlags. So I altered the tests too to make sure it doesn't happen again. Previously the qemuxml2xmloutdata was a softlink to the source qemuxml2argvdata, so I unlinked and recreated the output file to force generation of the adddress. Without the test changes, an address generation returns: libvirt: Domain Config error : internal error: Cannot automatically add a new PCI bus for a device with connect flags 00 if an address was supplied in the test, a restart of libvirtd or edit of a guest would display the following opaque message: warning : qemuDomainCollectPCIAddress:1237 : qemuDomainDeviceCalculatePCIConnectFlags() thinks that the device with PCI address 0000:00:09.0 should not have a PCI address where the address is related to the guest PCI address provided. 2017-12-13 Marek Marczykowski-Górecki xenconfig: fix issue found by coverity in multiple IP support virStringSplit may return NULL, so we must handle that. Cc: John Ferlan 2017-12-13 Marek Marczykowski-Górecki xenconfig: Remove unnecessary variable clear in xenMakeIPList Remove the unnecessary clearing of address_array as VIR_ALLOC_N initialized the array already. Cc: John Ferlan 2017-12-13 John Ferlan qemu: Tolerate storage source private data being NULL for hotplug SCSI hostdev Commit id 'c5c96545' neglected to validate that the srcPriv was non-NULL before dereferencing. Similar problem to what was fixed by commit id '8056721c' but missed during multiple rebases and code reworks. storage: Reduce need for using storageDriverLock Now that the storage pools are self locking, we can reduce the number of places that need to take the big hammer storage driver lock 2017-12-13 John Ferlan storage: Convert virStoragePoolObjList to use virObjectRWLockable Now that we have a private storage pool list, we can take the next step and convert to using objects. In this case, we're going to use RWLockable objects (just like every other driver) with two hash tables for lookup by UUID or Name. Along the way the ForEach and Search API's will be adjusted to use the related Hash API's and the various FindBy functions altered and augmented to allow for HashLookup w/ and w/o the pool lock already taken. After virStoragePoolObjRemove we will need to virObjectUnref(obj) after to indicate the caller is "done" with it's reference. The Unlock occurs during the Remove. The NumOf, GetNames, and Export functions all have their own callback functions to return the required data and the FindDuplicate code can use the HashSearch function callbacks. 2017-12-13 John Ferlan storage: Privatize virStoragePoolObjListPtr Move the structure into virstorageobj.c. Use the virStoragePoolObjListNew allocator to fill in the @pools for the storage driver and test driver. 2017-12-13 John Ferlan storage: Fix path check in storagePoolLookupByTargetPath Commit id '5ab746b8' introduced the function as perhaps a copy of storageVolLookupByPath; however, it did not use the @cleanpath variable even though it used the virFileSanitizePath. So in essance the only "check" being done for failure is whether it was possible to strdup the path. Looking at the virStoragePoolDefParseXML one will note that the target.path is stored using the result of virFileSanitizePath. Therefore, this function should sanitize and use the input @path for the argument to storagePoolLookupByTargetPathCallback which is comparing against stored target.path values. Additionally, if there was an error we should use the proper error of VIR_ERR_NO_STORAGE_POOL (instead of VIR_ERR_NO_STORAGE_VOL). 2017-12-13 Marc Hartmayer audit: Log only an info message if audit_level < 2 and audit is not supported Replace the error message during startup of libvirtd with an info message if audit_level < 2 and audit is not supported by the kernel. Audit is not supported by the current kernel if the kernel does not have audit compiled in or if audit is disabled (e.g. by the kernel cmdline). Reviewed-by: Boris Fiuczynski 2017-12-12 Daniel P. Berrange virsh: fix typo s/node filters/network filters/ Update to latest keycodemapdb content Reviewed-by: Andrea Bolognani 2017-12-12 Michal Privoznik fillQemuCaps: Don't leak machine string ==25251== 5 bytes in 1 blocks are definitely lost in loss record 7 of 81 ==25251== at 0x4C2BEDF: malloc (vg_replace_malloc.c:299) ==25251== by 0x967E379: strdup (in /lib64/libc-2.25.so) ==25251== by 0x5366F9F: virStrdup (virstring.c:941) ==25251== by 0x538BF1D: virDomainCapsNew (domain_capabilities.c:121) ==25251== by 0x10EACE: test_virDomainCapsFormat (domaincapstest.c:295) ==25251== by 0x10FBD2: virTestRun (testutils.c:180) ==25251== by 0x10F192: mymain (domaincapstest.c:457) ==25251== by 0x111C7F: virTestMain (testutils.c:1119) ==25251== by 0x10FA3C: main (domaincapstest.c:528) Reviewed-by: John Ferlan 2017-12-12 Michal Privoznik xenMakeIPList: Don't leak @address_array ==32171== 32 bytes in 1 blocks are definitely lost in loss record 44 of 107 ==32171== at 0x4C2DEF6: calloc (vg_replace_malloc.c:711) ==32171== by 0x55744A9: virAllocN (viralloc.c:191) ==32171== by 0x12CED2: xenMakeIPList (xen_common.c:1186) ==32171== by 0x12D0BE: xenFormatNet (xen_common.c:1221) ==32171== by 0x12F0D2: xenFormatVif (xen_common.c:1889) ==32171== by 0x12F2B4: xenFormatConfigCommon (xen_common.c:1944) ==32171== by 0x13BA32: xenFormatXL (xen_xl.c:1971) ==32171== by 0x1186CA: testCompareParseXML (xlconfigtest.c:105) ==32171== by 0x118A64: testCompareHelper (xlconfigtest.c:205) ==32171== by 0x119E36: virTestRun (testutils.c:180) ==32171== by 0x11970E: mymain (xlconfigtest.c:301) ==32171== by 0x11BEE3: virTestMain (testutils.c:1119) Reviewed-by: John Ferlan 2017-12-12 Michal Privoznik xenParseXLVnuma: Don't leak @tmp and @token ==30399== 180 (144 direct, 36 indirect) bytes in 3 blocks are definitely lost in loss record 91 of 111 ==30399== at 0x4C2E0FF: realloc (vg_replace_malloc.c:785) ==30399== by 0x5574572: virReallocN (viralloc.c:245) ==30399== by 0x5574668: virExpandN (viralloc.c:294) ==30399== by 0x55747AB: virResizeN (viralloc.c:352) ==30399== by 0x560074D: virStringSplitCount (virstring.c:115) ==30399== by 0x137A59: xenParseXLVnuma (xen_xl.c:442) ==30399== by 0x13952B: xenParseXL (xen_xl.c:1064) ==30399== by 0x11884D: testCompareFormatXML (xlconfigtest.c:152) ==30399== by 0x118A87: testCompareHelper (xlconfigtest.c:207) ==30399== by 0x119E36: virTestRun (testutils.c:180) ==30399== by 0x119186: mymain (xlconfigtest.c:274) ==30399== by 0x11BEE3: virTestMain (testutils.c:1119) Reviewed-by: John Ferlan 2017-12-12 Yuri Chornoivan Fix minor typos 2017-12-11 Jiri Denemark qemu: Avoid comparing size_t with -1 ncpus would be -1 on error and the cleanup for loop would not be skipped in this case. Reviewed-by: John Ferlan 2017-12-11 Michal Privoznik qemuDomainUndefineFlags: Fix error message https://bugzilla.redhat.com/show_bug.cgi?id=1522706 If domain is active, but the undefine API was called without the VIR_DOMAIN_UNDEFINE_KEEP_NVRAM flag set, the following incorrect error message is produced: error: Requested operation is not valid: cannot delete inactive domain with nvram 2017-12-09 Lin Ma qemu: Introduce qemuDomainDeviceDefValidateControllerIDE Move the IDE controller check from command line building to controller def validation. Also explicitly include the avoidance check for the implicit IDE controller from qemuBuildSkipController. Cause the IDE case for command line building to generate a failure if called to add an IDE since that shouldn't happen if the Validate code did the right thing. 2017-12-09 John Ferlan qemu: Move CCW S390 Address check to controller def validate Move the call to qemuDomainCheckCCWS390AddressSupport from qemuBuildControllerDevStr to qemuDomainDeviceDefValidateController. This means we will get the qemuCaps from the driver opaque variable passed to qemuDomainDeviceDefValidate. 2017-12-09 John Ferlan qemu: Introduce qemuDomainDeviceDefValidateController Introduce the bare bones helper to validate whether the controller definition is valid. qemu: Introduce qemuBuildSkipController Move the non USB implicit controller checks into their own helper to reduce the cruft in qemuBuildControllerDevCommandLine. 2017-12-08 Marek Marczykowski-Górecki tests: add test for multiple IPs for libxl and xenconfig driver Test conversion of multiple IP addresses to/from xl format and domXML. Also test libxl_domain_config generator handling of multiple IP addresses. Reviewed-by: Jim Fehlig 2017-12-08 Marek Marczykowski-Górecki xenconfig: add support for multiple IP addresses Xen's xl config format has long supported specifying multiple IP addresses for virtual interfaces. E.g. vif = [ "ip=10.0.0.1 10.1.1.1 2000::1, ..." ] Add support for converting multiple IP addresses to/from domXML. Reviewed-by: Jim Fehlig 2017-12-08 Marek Marczykowski-Górecki libxl: add support for multiple IP addresses vif-* scripts support it for a long time, and expect addresses to be separated by spaces. Add appropriate support to libxl driver. Reviewed-by: Jim Fehlig 2017-12-08 Jiri Denemark migration.html: Clarify configuration file handling docs Migration never removes any configuration files on the destination host. Thus when the domain is already defined on the destination, it will stay persistent even after migration without --persist. https://bugzilla.redhat.com/show_bug.cgi?id=1514930 2017-12-08 Jiri Denemark qemu: Make sure host-model uses CPU model supported by QEMU When reconnecting to a running domain started by old libvirt, which did not change host-model into a custom CPU definition, we replace the CPU definition with a specific CPU model from host capabilities. However, that CPU model may not be supported by the running qemu process. We need to translate the CPU model to one of the models which libvirt could have used when starting the domain. https://bugzilla.redhat.com/show_bug.cgi?id=1521202 2017-12-08 Jiri Denemark qemu: Separate fetching CPU definitions from filling qemuCaps virQEMUCapsProbeQMPCPUDefinitions is now a small wrapper which fills in qemuCaps with CPU models fetched by virQEMUCapsFetchCPUDefinitions. 2017-12-08 Peter Krempa qemu: blockjob: Reset disk source index after pivot Since we are re-detecting the backing chain after pivoting to the active block commit target (or block copy target) the disk index needs to be reset to 0. This is necessary since we move a member of the backing chain to disk->src but clear indexes only starting from disk->src->backingStore. The freshly detected images have indexes starting from 1, but since we've pivoted into an image which was previously a backing store it would have a non-0 index. The lookup function would then return the top of the chain for queries like 'vda[1]' instead of the first backing store. This problem will not be present once we keep the disk indexes stable. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1519745 2017-12-07 Ján Tomko qemuDomainNamespaceSetupDisk: initialize npaths Introduced by commit d3db304. Reported-by: John Ferlan 2017-12-07 Peter Krempa docs: domain: Fix documentation of the 'snapshot' attribute for Emphasise the valid values by wrapping them in and reword the last sentence so that the invalid value example can be dropped. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1523070 2017-12-07 Chen Hanxiao virerror: mark VIR_ERR_AGENT_UNSYNCED as DEPRECATED Since commit 5e5019bf, we've no longer use VIR_ERR_AGENT_UNSYNCED anymore. Mark it as DEPRECATED. 2017-12-07 Peter Krempa storage: Don't dereference driver object if virStorageSource is not initialized virStorageFileReportBrokenChain uses data from the driver private data pointer to print the user and group. This would lead to a crash in call paths where we did not initialize the storage backend as recently added in commit 24e47ee2b93 to qemuDomainDetermineDiskChain. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1522682 2017-12-07 Daniel P. Berrange docs: remove outdated link to Fedora mingw staging repo The Fedora mingw support is all merged in Fedora repos, so remove the outdated link. Reviewed-by: John Ferlan 2017-12-07 Daniel P. Berrange docs: update entries in the apps page Change all links to https:// where the remote site supports it. Fix URLs for a few packages that moved, and delete entries which appear to be dead. Reviewed-by: John Ferlan 2017-12-07 Daniel P. Berrange docs: update instructions for TLS cert generation Currently we only describe setting the CN field for server certs. This leads to inevitable pain for users who set it to the fully qualified hostname and then use a unqualified hostname or IP address to connect in the URI. Describe the usage of Subject Alt Name extensions, to provide multiple hostnames and IP addresses. This will help users avoid the classic mistake and is important future proofing, since at least in browsers, TLS libraries no longer use the CN field for validation, mandating use of SAN info instead. Reviewed-by: John Ferlan 2017-12-07 Daniel P. Berrange nwfilter: remove bogus 'protocolid' attribute on arp/rarp fields Various example XML documents for arp/rarp filtering have a protocolid XML attribute defined. This is never parsed or output by the libvirt XML handling code, so shouldn't be present in example XML files either Reviewed-by: John Ferlan 2017-12-07 Daniel P. Berrange rng: fix nwfilter rule contents The contents of a are a choice of exactly one union member. The RNG schema, however, was allowing an arbitrary number of instances of every union member at once. Reviewed-by: John Ferlan 2017-12-07 Lin Ma tests: Drop IDE controller in CCW Adding an IDE controller for a machinetype that has no built-in IDE controller, libvirt will log an error. Currently the machinetype list which returns by qemuDomainMachineHasBuiltinIDE only includes 440fx, malta, sun4u and g3beige. 2017-12-07 Lin Ma tests: Remove use of IDE disk for pseries floppy test Adding an IDE controller for a machinetype that has no built-in IDE controller, libvirt will log an error. Currently the machinetype list which returns by qemuDomainMachineHasBuiltinIDE only includes 440fx, malta, sun4u and g3beige. Remove the disk and the .args file since the expectation is the test will fail in qemuxml2argvtest because floppy is not supported on pseries and thus no disk is necessary and no .args file would be created to compare against. 2017-12-07 John Ferlan qemu: Use virDomainControllerType in qemuBuildControllerDevStr switch Make sure all types of virDomainControllerType are handled in the switch statement. 2017-12-06 Ján Tomko Do not pass driver to qemuDomainNamespace{Setup,Teardown} functions The underlying function which needs the driver gets it from the passed virDomainObj object anyway. Introduce qemuDomainNamespaceUnlinkPath Use it in every qemuDomainNamespaceTeardown* function that only wants to unlink one device. Introduce qemuDomainNamespaceUnlinkPaths Split out the logic of unlinking devices from qemuDomainNamespaceTeardownHostdev for reuse in other functions. Introduce qemuDomainNamespaceMknodPath Use this function in qemuDomainNamespaceSetup* functions which only require creating one device. 2017-12-06 Ján Tomko Introduce qemuDomainNamespaceMknodPaths Separate the logic of creating devices from their gathering. Use this new function in qemuDomainNamespaceSetupHostdev and qemuDomainNamespaceSetupDisk. 2017-12-06 Ján Tomko qemuDomainNamespaceSetupHostdev: rename path to paths To match the "things/nthings" pattern used in virDomainDef. qemuDomainNamespaceTeardownHostdev: rename path to paths To match the "things/nthings" pattern used in virDomainDef. 2017-12-06 Nikolay Shirokovskiy qemu: report drive mirror errors on migration qemu: prepare blockjob complete event error usage This patch pass event error up to the place where we can use it. Error is passed only for sync blockjob event mode as we can't use the error in async mode. In async mode we just pass the event details to the client thru event API but current blockjob event API can not carry extra parameter. 2017-12-06 Daniel P. Berrange nwfilter: don't crash listing filters in unprivileged daemon The unprivileged libvirtd does not support nwfilter config, by leaves the driver active. It is supposed to result in all APIs being an effective no-op, but several APIs rely on driver->nwfilters being non-NULL, or they will reference a NULL pointer. Rather than adding checks for NULL in many places, just make sure driver->nwfilters is always initialized. Reviewed-by: John Ferlan 2017-12-05 Michal Privoznik test: Drop useless prefix for genericxml2xml test data There's no reason for the files to have generic- prefix since they all live under genericxml2xmlindata and genericxml2xmloutdata directories. test: Drop useless prefix for qemuxml2xmldata test data There's no reason for the files to have qemuxml2xmlout- prefix since they all live under qemuxml2xmloutdata directory. test: Drop useless prefix for qemuargv2xml test data There's no reason for the files to have qemuargv2xml- prefix since they all live under qemuargv2xmldata directory. test: Drop useless prefix for qemuagent test data There's no reason for the files to have qemuagent- prefix since they all live under qemuagentdata directory. 2017-12-05 Pavel Hrdina qemu: fix security labeling for attach/detach of char devices Commit e93d844b90 was not enough to fix the permission denied issue. We need to apply security labels as well. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1465833 2017-12-05 Pavel Hrdina security: introduce virSecurityManager(Set|Restore)ChardevLabel SELinux and DAC drivers already have both functions but they were not exported as public API of security manager. 2017-12-05 Andrea Bolognani travis: Don't try to install brew packages twice gettext, gnutls and libgcrypt are already installed on the system, so we don't need to request their installation. Reviewed-by: Daniel P. Berrange 2017-12-05 Andrea Bolognani travis: Upgrade brew packages Installed packages might be outdated by the time the build runs, so we should update them. Reviewed-by: Daniel P. Berrange 2017-12-05 Andrea Bolognani util: gettid() is Linux-specific The manual page clearly states that gettid() is Linux-specific and should not be used in programs that are intended to be portable. Unfortunately, it looks like macOS implemented the functionality and defined SYS_gettid accordingly, only to deprecate syscall() altogether with 10.12 (Sierra), released last late year. To avoid compilation errors, call gettid() on Linux only. Reviewed-by: Daniel P. Berrange 2017-12-05 Andrea Bolognani maint: Bump version number to 4.0.0 As documented in https://libvirt.org/downloads.html#schedule https://libvirt.org/downloads.html#numbering the next release will happen in the middle of January 2018 and, being the first release of a new year, will bring a brand new major version number with it. 2017-12-05 Michal Privoznik tests: Drop qemuxml2argv- prefix for qemuxml2argv test cases Similarly to the previous commit, rename .args files. The files were renamed using the following commands. From qemuxml2argvdata: for i in qemuxml2argv-*.args; do mv $i ${i#qemuxml2argv-}; done and then (to fix broken symlinks) from qemuxml2argvdata and qemuxml2xmloutdata: for i in $(find . -xtype l); do \ ln -sf $(readlink $i | sed 's/qemuxml2argv-//') $i; done 2017-12-05 Michal Privoznik tests: Drop qemuxml2argv- prefix for qemuxml2argv-*.xml test cases These XMLs live in a separate directory, there's no need for them to have a special prefix in addition. It also doesn't play nicely with ':e' completion in Vim, finding proper file based on qemuxml2argvtest.c is also needlessly complicated. The files were renamed using the following commands. From qemuxml2argvdata: for i in qemuxml2argv-*.xml; do mv $i ${i#qemuxml2argv-}; done and then (to fix broken symlinks) from qemuxml2argvdata and qemuxml2xmloutdata: for i in $(find . -xtype l); do \ ln -sf $(readlink $i | sed 's/qemuxml2argv-//') $i; done 2017-12-05 Michal Privoznik Post-release version bump to 3.11.0 2017-12-04 Daniel Veillard Release of libvirt 3.10.0 2017-12-04 Jim Fehlig apparmor: allow qemu abstraction to read /proc/pid/cmdline Noticed the following denial in audit.log when shutting down an apparmor confined domain type=AVC msg=audit(1512002299.742:131): apparmor="DENIED" operation="open" profile="libvirt-66154842-e926-4f92-92f0-1c1bf61dd1ff" name="/proc/1475/cmdline" pid=2958 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=469 ouid=0 Squelch the denial by allowing read access to /proc//cmdline. 2017-12-04 Daniel P. Berrange Refresh translations from zanata 2017-12-01 John Ferlan conf: Fix memory leak for distances in virDomainNumaFree Commit id '74119a03f' neglected to clean up @distances when the numa definition is cleaned up. conf: Clean up virDomainNumaDefCPUFormatXML Don't use a unary comparison for an int value - compare against zero directly instead. 2017-12-01 John Ferlan conf: Clean up virDomainNumaDefNodeDistanceParseXML Clean up the style a bit w/r/t to not using a unary operator on an integer value that could be zero - compare vs. zero instead. Set the def->mem_nodes[*].distances to rdist or ldist inside the if condition - no need to set outside since the value being set to is what was fetched. During cleanup, be sure to initialize the ndistances on error and use the < 0 comparison not the unary one. 2017-12-01 Michal Privoznik qemuStateInitialize: Don't leak @memoryBackingPath ==899== 39 bytes in 1 blocks are definitely lost in loss record 732 of 1,003 ==899== at 0x4C2AEDF: malloc (vg_replace_malloc.c:299) ==899== by 0x8B68CE7: vasprintf (in /lib64/libc-2.25.so) ==899== by 0x55498D2: virVasprintfInternal (virstring.c:708) ==899== by 0x55499E7: virAsprintfInternal (virstring.c:729) ==899== by 0x2BECFFF0: qemuGetMemoryBackingBasePath (qemu_conf.c:1757) ==899== by 0x2BF23225: qemuStateInitialize (qemu_driver.c:893) ==899== by 0x563073D: virStateInitialize (libvirt.c:770) ==899== by 0x124CC4: daemonRunStateInit (libvirtd.c:834) ==899== by 0x55521CD: virThreadHelper (virthread.c:206) ==899== by 0x88D9686: start_thread (in /lib64/libpthread-2.25.so) ==899== by 0x8BEAEFE: clone (in /lib64/libc-2.25.so) virDomainDiskBackingStoreParse: Don't leak @idx ==1277== 8 bytes in 4 blocks are definitely lost in loss record 39 of 131 ==1277== at 0x4C2AEDF: malloc (vg_replace_malloc.c:299) ==1277== by 0x68BBBC8: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4) ==1277== by 0x53B1DC2: virXMLPropString (virxml.c:510) ==1277== by 0x53D696A: virDomainDiskBackingStoreParse (domain_conf.c:8639) ==1277== by 0x53DA684: virDomainDiskDefParseXML (domain_conf.c:9590) ==1277== by 0x53F619F: virDomainDefParseXML (domain_conf.c:19233) ==1277== by 0x53F96EE: virDomainDefParseNode (domain_conf.c:20083) ==1277== by 0x53F9540: virDomainDefParse (domain_conf.c:20027) ==1277== by 0x53F95E6: virDomainDefParseFile (domain_conf.c:20053) ==1277== by 0x44D1D4: testCompareDomXML2XMLFiles (testutils.c:1265) ==1277== by 0x42FC7C: testXML2XMLActive (qemuxml2xmltest.c:71) ==1277== by 0x44AD20: virTestRun (testutils.c:180) virDomainDiskSourceNetworkParse: Don't leak @tlsCfg or @haveTLS ==861== 3 bytes in 1 blocks are definitely lost in loss record 3 of 168 ==861== at 0x4C2AEDF: malloc (vg_replace_malloc.c:299) ==861== by 0x8C7FBC8: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4) ==861== by 0x5DCCDC2: virXMLPropString (virxml.c:510) ==861== by 0x5DF1232: virDomainDiskSourceNetworkParse (domain_conf.c:8445) ==861== by 0x5DF1728: virDomainDiskSourceParse (domain_conf.c:8576) ==861== by 0x5DF41A5: virDomainDiskDefParseXML (domain_conf.c:9238) ==861== by 0x5E1119F: virDomainDefParseXML (domain_conf.c:19233) ==861== by 0x5E146EE: virDomainDefParseNode (domain_conf.c:20083) ==861== by 0x5E14540: virDomainDefParse (domain_conf.c:20027) ==861== by 0x5E145E6: virDomainDefParseFile (domain_conf.c:20053) ==861== by 0x4053CC: testCompareXMLToArgv (qemuxml2argvtest.c:455) ==861== by 0x41F135: virTestRun (testutils.c:180) 2017-11-30 Peter Krempa qemu: domain: Fix backing store terminator for non-backing local files Raw local files do not pass through the backing store detector and thus the code did not allocate the required backing store terminator for them. Previously the terminating element would be formatted into the XML since the default values used for the metadata allowed that. This is a regression since a693fdba0111ff which was not detected in the review. This patch also reverts all the changes in the test files. 2017-11-30 Peter Krempa qemu: process: Move handling of non-backing files into qemuDomainDetermineDiskChain Until now we would skip loading of the backing chain for files which don't support backing chains only when starting up the VM. Move the check from qemuProcessPrepareHostStorage with some adaptations so that's always applied. qemu: domain: Refactor control flow in qemuDomainDetermineDiskChain Split out clearing of the backing chain prior to other code since it will be required later and optimize few layers of nested conditions and loops. 2017-11-30 Pavel Hrdina conf: fix migratable XML for graphics if socket is generated based on config The graphics code is complex and there are a lot of exceptions and backward compatible combinations. One of them is the possibility to configure "spice_auto_unix_socket" in qemu.conf which will convert all spice graphics with listen type "address" without any address specified to listen type "socket" when the guest is started. We don't format this generated socket into migratable XML to make migration work with older libvirt. However, spice has another exception that if autoport='no' and there is no port configured it is converted to listen type "none". Because of this we need to format autoport='yes' to make sure that the listen type will be the same as the offline XML. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1511407 Reviewed-by: Jiri Denemark 2017-11-30 Ján Tomko virQEMUCapsHasPCIMultiBus: assume true if we have no version information In status XML, we do not store the QEMU version information, we only format all the capabilities. We dropped QEMU_CAPS_PCI_MULTIBUS in commit 5b783379 which was released in libvirt 3.2.0. Therefore the only way of telling if the already running domain at the time of daemon restart has been started with a QEMU that does use 'pci.0' or not on PPC is to look at the pci-root controller's alias. This is not an option if the domain has a user-specified alias for the pci-root. Instead of reintroducing the capability, assume 'pci.0' when we have no version information. That way the only left broken use case would be the combination of user aliases and very old QEMU. Partially reverts commit 3a37af1e4. https://bugzilla.redhat.com/show_bug.cgi?id=1518148 2017-11-30 Ján Tomko virQEMUCapsHasPCIMultiBus: use def->os.arch We do not fill out qemuCaps->arch when parsing status XML. Use def->os.arch like we do for PPC. This fixes hotplug after daemon restart for domains that use a user alias for the implicit pci-root on x86. https://bugzilla.redhat.com/show_bug.cgi?id=1518148 2017-11-30 Ján Tomko qemu: prefer the PCI bus alias from status XML For some corner cases, virQEMUCapsHasPCIMultiBus depends on the QEMU version, which is by design not stored in the status XML and therefore it cannot be fixed for all existing running domains. Prefer the controller alias read from the status XML when formatting PCI addresses and only fall back to using virQEMUCapsHasPCIMultiBus if the alias is a user alias. This fixes hotplug after daemon restart for domains not using user aliases. Partially reverts commit 937f3195. https://bugzilla.redhat.com/show_bug.cgi?id=1518148 2017-11-30 Ján Tomko Introduce virDomainDeviceAliasIsUserAlias Allow parts of code outside domain_conf to decide whether the alias is user-specified or not. 2017-11-30 Daniel P. Berrange Remove non-existant 'wiremode' attribute The 'wiremode' attribute exists in a couple of Xen XML files, but no code has ever parsed that value. It was later added to the RNG schema too, again despite there not being any code which parses it. Reviewed-by: Jim Fehlig 2017-11-30 Daniel P. Berrange Misc XML schema fixes for libxl The libxlxml2domconfigdata directory was not covered in the RNG schema tests. This hid a few bugs in both the libxl XML files and the RNG schema itself. Reviewed-by: Jim Fehlig 2017-11-29 Boris Fiuczynski qemu-capabilities: Adjust function header comments of virQEMUCapsInitCPUModel* Adjust function descriptions of virQEMUCapsInitCPUModelS390 and virQEMUCapsInitCPUModel to the changes introduced with commitID 74fc32a955. Reviewed-by: Marc Hartmayer 2017-11-29 Jiri Denemark spec: Make the build reproducible If the spec file applies a patch which touches any file in the API XMLs dependency tree, we need to regenerate the XMLs and consequently recreate hvsupport.html. The file will contain a time stamp in a comment which means it will be different every time the package is built. The commit a54c9622860 which added the time stamp also added support for SOURCE_DATE_EPOCH environment variable. Let's set it to the time stamp of the spec file itself to make the build reproducible. Reviewed-by: Pino Toscano 2017-11-29 Jiri Denemark spec: Turn on verbose build When building a package in a build system, such as koji or cbs, logs are the only thing which can be used to diagnose failures. Make them verbose since human friendly output of V=0 build doesn't really help when a build fails. Reviewed-by: Pino Toscano 2017-11-28 Andrea Bolognani news: Update for serial console fixes Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Require QEMU_CAPS_DEVICE_PL011 for pl011 Even though we never format the device on the QEMU command line, as it's a platform serial device that's not user-instantiable, we should still make sure it's available before using it. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Add QEMU_CAPS_DEVICE_PL011 All serial devices shoule have an associated capability. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Require QEMU_CAPS_DEVICE_ISA_SERIAL for isa-serial We should make sure the isa-serial device is available before formatting it on the QEMU command line. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Add QEMU_CAPS_DEVICE_ISA_SERIAL All serial devices shoule have an associated capability. Reviewed-by: Pavel Hrdina 2017-11-28 Pino Toscano qemu: switch s390/s390x default console back to serial Now that and on s390/s390x behave a bit more like the other architectures, remove this extra differentation, and use sclp console by default for new guests. New virtio consoles can still be added, and it is actually needed because of the limited number of instances for sclp and sclplm. This reverts commit b1c88c14764e0b043a269d454a83a6ac7af34eac, whose reasons are not totally clear. Reviewed-by: Andrea Bolognani Reviewed-by: Bjoern Walk 2017-11-28 Pino Toscano conf: add VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_SCLP Introduce specific a target types with two models for the console devices (sclp and sclplm) used in s390 and s390x guests, so isa-serial is no more used for them. This makes usable on s390 and s390x guests, with at most only a single sclpconsole and one sclplmconsole devices usable in a single guest (due to limitations in QEMU, which will enforce already at runtime). Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1449265 Reviewed-by: Andrea Bolognani Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Add target type and model for pl011 We can finally introduce a specific target model for the pl011 device used by mach-virt guests, which means isa-serial will no longer show up to confuse users. We make sure migration works in both directions by interpreting the isa-serial target type, or the lack of target type, appropriately when parsing the guest XML, and skipping the newly-introduced type when formatting if for migration. We also verify that pl011 is not used for non-mach-virt guests and add a bunch of test cases. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=151292 Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Support usb-serial and pci-serial on pSeries The existing implementation set the address type for all serial devices to spapr-vio, which made it impossible to use other devices such as usb-serial and pci-serial; moreover, some decisions were made based on the address type rather than the device type. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1512934 Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Add target type and model for spapr-vty We can finally introduce a specific target model for the spapr-vty device used by pSeries guests, which means isa-serial will no longer show up to confuse users. We make sure migration works in both directions by interpreting the isa-serial target type, or the lack of target type, appropriately when parsing the guest XML, and skipping the newly-introduced type when formatting if for migration. We also verify that spapr-vty is not used for non-pSeries guests and add a bunch of test cases. This commit is best viewed with 'git show -w'. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1511421 Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Remove redundancy in qemuBuildSerialChrDeviceStr() Instead duplicating the capability check for each possible target model, introduce a small helper that matches the target model with the corresponding capability and collapse all existing checks into a single one. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Format targetModel for serial devices Now that we've created a distinction between target type and target model, with the latter being the concrete device name, it's time to switch to formatting the model instead of the type. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Validate target model for serial devices Target model and target type must agree for the configuration to make sense, so check that's actually the case and error out otherwise. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Set targetModel based on targetType for serial devices Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Parse and format virDomainChrSerialTargetModel This information will be used to select, and store in the guest configuration in order to guarantee ABI stability, the concrete (hypervisor-specific) model for serial devices. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Improve qemuDomainChrTargetDefValidate() Instead of validating each target type / address type combination separately, create a small helper to perform the matching and collapse all existing checks into a single one. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Introduce qemuDomainChrTargetDefValidate() Instead of waiting until we get to command line generation, we can validate the target for a char device much earlier. Move all the checks out of qemuBuildSerialChrDeviceStr() and into the new fuction. This will later allow us to validate the target for platform devices. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Remove ATTRIBUTE_FALLTHROUGH from virDomainChrTargetDefFormat() Formatting the element for serial devices will become a bit more complicated later on, and leaving the fallthrough behavior there would do nothing but complicate it further. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Improve virDomainChrTargetDefFormat() Make the switch statement type-aware, avoid calling virDomainChrTargetTypeToString() more than once and check its return value before using it. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Check virDomainChrSourceDefFormat() return value The function can fail, but none of the caller were accounting for that. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Improve error handling in virDomainChrDefFormat() We don't need to store the return value since we never modify it. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Introduce virDomainChrTargetDefFormat() Move formatting of the element for char devices out of virDomainChrDefFormat() and into its own function. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Drop virDomainChrDeviceType.targetTypeAttr This attribute was used to decide whether to format the type attribute of the element, but the logic didn't take into account all possible cases and as such could lead to unexpected results. Moreover, it's one more thing to keep track of, and can easily fall out of sync with other attributes. Now that we have VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_NONE, we can use that value to signal that no specific target type has been configured for the serial device and as such the attribute should not be formatted at all. All other values are now formatted. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Introduce VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_NONE This is the first step in getting rid of the assumption that isa-serial is the default target type for serial devices. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani conf: Run devicePostParse() again for the first serial device The devicePostParse() callback is invoked for all devices so that drivers have a chance to set their own specific values; however, virDomainDefAddImplicitDevices() runs *after* the devicePostParse() callbacks have been invoked and can add new devices, in which case the driver wouldn't have a chance to customize them. Work around the issue by invoking the devicePostParse() callback after virDomainDefAddImplicitDevices(), only for the first serial devices, which might have been added by it. The same was already happening for the first video device for the very same reason. This will become important later on, when we will change virDomainDefAddConsoleCompat() not to set a targetType for automatically added serial devices. Reviewed-by: Pavel Hrdina 2017-11-28 Andrea Bolognani qemu: Introduce qemuDomainChrDefPostParse() Having a separate function for char device handling is better than adding even more code to qemuDomainDeviceDefPostParse(). Reviewed-by: Pavel Hrdina Reviewed-by: Marc Hartmayer 2017-11-28 Andrea Bolognani docs: Improve documentation for serial consoles Our current documentation is missing some information and doesn't do a great job at explaining how the and elements are connected. Let's try to fix that. Reviewed-by: Pavel Hrdina 2017-11-27 Peter Krempa news: Document qemu image locking 2017-11-27 Julio Faracco virsh: Fix virsh prompt when connection changes to readonly mode. This commit fixes the virsh prompt when reconnection to the same URI is called: `virsh # connect --readonly` (Reconnect). The problem is happening because the code is considering URI (name) as a mandatory parameter to change the prompt. This commit remove the assignment into `priv->readonly` from `if (name)` conditional. Before: virsh # uri qemu:///system virsh # connect --readonly virsh # After: virsh # uri qemu:///system virsh # connect --readonly virsh > Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1507737 Reviewed-by: Erik Skultety 2017-11-24 John Ferlan docs: Add news article to describe iSCSI usage of secret object docs: Add news article regarding auth/encryption placement 2017-11-24 John Ferlan qemu: Use secret objects to pass iSCSI passwords https://bugzilla.redhat.com/show_bug.cgi?id=1425757 The blockdev-add code provides a mechanism to sanely provide user and password-secret arguments for iscsi without placing them on the command line to be viewable by a 'ps -ef' type command or needing to create separate -iscsi devices for each disk/volume found. So modify the iSCSI command line building to check for the presence of the capability in order properly setup and use the domain master secret object to encrypt the password in a secret object and alter the parameters for the command line to utilize. Modify the xml2argvtest to exhibit the syntax for both disk and hostdev configurations. 2017-11-24 John Ferlan qemu: Get capabilities to use iscsi password-secret argument Detect the capability via the query-qmp-schema for blockdev-add to find the 'password-secret' parameter that will allow the iSCSI code to use the master secret object to encrypt the secret for an and only need to provide the object id of the secret on the command line thus obsfuscating the passphrase. qemu: Refactor qemuBuildSCSIiSCSIHostdevDrvStr slightly Rather than building the "file" string in qemuBuildSCSIHostdevDrvStr build it in the called helper. qemu: Remove private hostdev Since it's not longer used to shuttle the @secinfo, let's remove the private hostdev completely. qemu: Use private storage source for iscsi instead of private hostdev Rather than placing/using privateData about secinfo in the hostdev, let's use the virStorageSource private data instead. 2017-11-24 John Ferlan conf,qemu: Replace iscsisrc fields with virStorageSourcePtr Rather than picking apart the two pieces we need/want (path, hosts, and auth)- let's allocate/use a virStorageSourcePtr for iSCSI storage. The end result is that qemuBuildSCSIiSCSIHostdevDrvStr doesn't need to "fake" one for the qemuBuildNetworkDriveStr call. 2017-11-24 Ján Tomko qemu: Properly label and create evdev on input device hotplug Utilize all the newly introduced function to create the evdev node and label it on hotplug and destroy it on hotunplug. This was forgotten in commits bc9ffaf and 67486bb. https://bugzilla.redhat.com/show_bug.cgi?id=1509866 2017-11-24 Ján Tomko qemu: functions for dealing with input device namespaces and labels Introudce functions that will let us create the evdevs in namespaces and label the devices on input device hotplug/hotunplug. qemu: Introduce functions for input device cgroup manipulation Export qemuSetupInputCgroup and introduce qemuTeardownInputCgroup for hotunplug. security: Introduce functions for input device hot(un)plug Export the existing DAC and SELinux for separate use and introduce functions for stack, nop and the security manager. Introduce virDomainInputDefGetPath Use it to denadify qemuDomainSetupInput. 2017-11-24 Jason J. Herne s390: qemu-capabilities: Avoid error message when missing non-kvm host cpu info Libvirt prints an error on startup when it is missing host cpu model information for any queried qemu binary. On s390 we only have host cpu model information for kvm enabled qemu instances. So when virt type is not kvm, this is actually not an error on s390. This patch adds virt type as a parameter to virQEMUCapsInitCPUModelS390, and a new return code 2 for virQEMUCapsInitCPUModel and virQEMUCapsInitCPUModelS390. If the virt type is not kvm then we skip printing the scary error message and return 2 because this case is actually expected behavior. The new return code is meant to differentiate between the failure case and the case where we simply expect the cpu model information to be unattainable. Reviewed-by: Boris Fiuczynski Reviewed-by: Marc Hartmayer 2017-11-24 John Ferlan storage: Fix broken storage_driver build Commit id '5d5c732d7' had an incorrect assignment and was found by travis build: storage/storage_driver.c:1668:14: error: equality comparison with extraneous parentheses [-Werror,-Wparentheses-equality] if ((obj == virStoragePoolObjListSearch(&driver->pools, ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2017-11-24 John Ferlan storage: Convert virStoragePoolObj into virObjectLockable Now that we're moved the object into virstorageobj, let's make the code use the lockable object. storage: Introduce virStoragePoolObjListSearch Create an API to search through the storage pool objects looking for a specific truism from a callback API in order to return the specific storage pool object that is desired. storage: Introduce virStoragePoolObjListForEach Create an API to walk the pools->objs[] list in order to perform a callback function for each element of the objs array that doesn't care about whether the action succeeds or fails as the desire is to run the code over every element in the array rather than fail as soon as or if one fails. 2017-11-24 John Ferlan storage: Introduce virStoragePoolObjEndAPI For now it'll just call the virStoragePoolObjUnlock, but a future adjustment will do something different. Since the new API will check for a NULL object before the Unlock call, callers no longer need to check for NULL before calling. The virStoragePoolObjUnlock is now private/static to virstorageobj.c with a short term forward reference. 2017-11-24 John Ferlan nodedev: Restore setting of privileged Commit id '36555364' removed the setting of the driver->privileged, which the udevProcessPCI would need in order to read the PCI device configs. 2017-11-23 Peter Krempa qemu: command: Properly format disk 'debug' attribute Move the setup of the disk attribute to the disk source prepare function which will allow proper usage with JSON props and move the fallback (legacy) generating code into the block which is executed with legacy options. As a side-effect of this change we can clean up propagation of 'cfg' into the command generator. Also it's nice to see that the test output is the same even when the value is generated in a different place. 2017-11-23 Peter Krempa qemu: domain: Unify disk source prepare steps Aggregate setup of various aspects of a disk source (secrets, TLS, ...) into one function so that we don't need to call multiple across the code base. qemu: hotplug: Rename qemuDomainPrepareDisk to qemuHotplugPrepareDiskAccess Match the prefix of the file and choose a name which better describes what happens. qemu: block: Add support for formatting gluster debug level via JSON Improve the formatter so that we can use the 'debug' property straight away when using json. util: storage: Add fields for debug options for disk drivers Some drive backends allow output of debugging information which can be configured using properties of the image. Add fields to virStorageSource which will allow configuring them. 2017-11-23 Peter Krempa qemu: command: Inject password-secret only when not using JSON props The 'file.password-secret' injection should be used only if we are using the old formatter. When formatting the source string from the JSON properties, the property should be added there. Also drop the comment which refers to stuff that will not be used in libvirt since -blockdev is the way to go. 2017-11-23 Michal Privoznik tools: Work around ancient readline My latest commit of a785186446de785d uncovered a problem we fixed in 9eb23fe2 but then reverted in 834c5720e443. Turns out, some systems (I'm looking at you OS X) have ancient readline with broken header file. 2017-11-23 Peter Krempa qemu: command: Mark disks as such in qemu Qemu has now an internal mechanism for locking images to fix specific cases of disk corruption. This requires libvirt to mark the image as shared so that qemu lifts certain restrictions. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1378242 2017-11-23 Peter Krempa qemu: caps: Add capability for 'share-rw' disk option 'share-rw' for the disk device configures qemu to allow concurrent access to the backing storage. The capability is checked in various supported disk frontend buses since it does not make sense to partially backport it. 2017-11-23 Peter Krempa qemu: Disallow pivot of shared disks to unsupported storage Pivoting to a unsupported storage type might break the assumption that shared disks will not corrupt metadata. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1511480 2017-11-23 Peter Krempa qemu: snapshot: Disallow snapshot of unsupported shared disks Creating a snapshot would introduce a possibly unsupported member for sharing into the backing chain. Add a check to prevent that from happening. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1511480 2017-11-23 Peter Krempa qemu: domain: Reject shared disk access if backing format does not support it Disk sharing between two VMs may corrupt the images if the format driver does not support it. Check that the user declared use of a supported storage format when they want to share the disk. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1511480 2017-11-23 Peter Krempa qemu: block: Add function to check if storage source allows concurrent access Storage source format backing a shared device (e.g. running a cluster filesystem) needs to support the sharing so that metadata are not corrupted. Add a central function for checking this. qemu: domain: Refactor domain device validation function Use a style that will discourage from adding inline checks. qemu: domain: Move video device validation into separate function qemu: domain: Move hostdev validation into separate function qemu: domain: Despaghettify qemuDomainDeviceDefValidate Move network device validation into a separate function. qemu: Move snapshot disk validation functions into one Move the code so that both the new image and old image can be verified in the same function. 2017-11-23 Julio Faracco tests: changing network interface types when backend tag is defined. Some test cases have the backend tag inside wrong interfaces. The backend xml tag does not support . So this commit changes some network types inside the interfaces that have backend defined. 2017-11-23 Michal Privoznik news: Document which drivers support NUMA distances Reviewed-by: John Ferlan 2017-11-23 Michal Privoznik qemu: Support setting NUMA distances Since we already have such support for libxl all we need is qemu driver adjustment. And a test case. Reviewed-by: John Ferlan 2017-11-23 Michal Privoznik qemu_capabilities: Introcude QEMU_CAPS_NUMA_DIST This capability says if qemu is capable of specifying distances between NUMA nodes on the command line. Unfortunately, there's no real way to check this and thus we have to go with version check. QEMU introduced this in 0f203430dd8 (and friend) which was released in 2.10.0. Reviewed-by: John Ferlan 2017-11-23 Michal Privoznik numa: Introduce virDomainNumaNodeDistanceIsUsingDefaults The function returns true/false depending on distance configuration being present in the domain XML. Reviewed-by: John Ferlan 2017-11-23 Michal Privoznik virDomainNumaGetNodeDistance: Fix input arguments validation There's no point in checking if numa->mem_nodes[node].ndistances is set if we check for numa->mem_nodes[node].distances. However, it makes sense to check if the sibling node (@cellid) caller passed falls within boundaries. Reviewed-by: John Ferlan 2017-11-23 Jiri Denemark qemu: Use the end of QEMU log for reporting errors When QEMU dies, we read its output stored in a log file and use it for reporting a hopefully useful error. However, virReportError will trim the message to (VIR_ERROR_MAX_LENGTH - 1) characters, which means the end of the log (which likely contains the error message we want to report) may get lost. We should trim the beginning of the log instead. https://bugzilla.redhat.com/show_bug.cgi?id=1335534 Reviewed-by: Pavel Hrdina 2017-11-23 Jiri Denemark vierror: Define VIR_ERROR_MAX_LENGTH macro And use it instead of a magic 1024 constant. Reviewed-by: Pavel Hrdina 2017-11-23 Jiri Denemark qemu: Properly skip "char device redirected to" in QEMU log When reading QEMU log for reporting it as an error message, we want to skip "char device redirected to" line. However, this string is not printed at the beginning of a line, which means STRPREFIX will never find it. Reviewed-by: Pavel Hrdina 2017-11-23 xinhua.Cao remote: Move the call to remoteClientFreePrivateCallbacks from FreeFunc to CloseFunc Still because of commit id 'fe8f1c8b' where we generate a REF for the Register and that's transparent to the consumer (e.g. how would they know they need to ensure that Deregister is called), thus the purpose of this patch is to find a way to Deregister if it's determined that the consumer hasn't by the time of the "last" REF we'd have. This solution to this problem is to alter the processing to have the remoteClientCloseFunc handle performing the Deregister calls instead of the remoteClientFreeFunc because there's no way FreeFunc would be called unless the Deregister was already called. 2017-11-23 xinhua.Cao remote: Extract common clearing of event callbacks of client private data Extract common clearing of event callbacks as remoteClientFreePrivateCallbacks. the common function also separation including the sysident handling. 2017-11-23 Martin Kletzander Adjust backslash alignment syntax-check We have a check for backslash alignment that checks for two blanks preceding a backslash. However there can be alignment done using a tabulator and in some cases one might be enough. There are none currently, but I found out that was the case before. at some point. so let's check for \t preceding the final backslash as well. util: Fix leak in virStringTrimOptionalNewline Do not access any data if strlen() == 0. 2017-11-22 Peter Krempa qemu: command: Anotate formatting of the frontend attributes with -drive Explain that certain attributes formatted with -drive are in fact attributes of the drive itself and not the storage backing it. qemu: command: Move formatting of disk io error policy from -drive That's a disk frontend attribute. Move the code to a separate function since it's non-trivial and call it from the frontend attribute formatter. qemu: command: Move disk trhottling argument building into a separate function Isolate it from the other code. qemu: command: Refactor logic when formatting -drive Move all logic depending on whether we are solely formatting -drive (no -device along with it) into one block. qemu: command: Move around order of generating -drive arguments Move together sections which are conditionaly executed depending on whether -device will be used together with the -drive. qemu: command: Move disk 'serial' into frontend parameter formatter Disk serial is not a property of the image but of the disk frontend. Account for this appropriately. tests: qemuxml2argv: Test SD card with serial number tests: qemuxml2xml: Run the 'disk-serial' test 2017-11-22 Peter Krempa qemu: command: Split out geometry frontend attribute formatting from -drive Historically we've formatted a lot of the attributes of a disk (disk geometry, etc) with -drive. Since we use -device now, they should be formatted there. Extract them to a separate function for keeping compatibility with SDcards which still use only -drive. Start this by moving the geometry into a separate function. 2017-11-22 Peter Krempa qemu: domain: Don't call namespace setup for storage already accessed by vm When doing block commit we need to allow write for members of the backing chain so that we can commit the data into them. qemuDomainDiskChainElementPrepare was used for this which since commit 786d8d91b4 calls qemuDomainNamespaceSetupDisk which has very adverse side-effects, namely it relabels the nodes to the same label it has in the main namespace. This was messing up permissions for the commit operation since its touching various parts of a single backing chain. Since we are are actually not introducing new images at that point add a flag for qemuDomainDiskChainElementPrepare which will refrain from calling to the namespace setup function. Calls from qemuDomainSnapshotCreateSingleDiskActive and qemuDomainBlockCopyCommon do introduce new members all calls from qemuDomainBlockCommit do not, so the calls are anotated accordingly. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1506072 2017-11-22 ZhiPeng Lu virsh: domifstat: enhance the command's help output According to the man page can be specified either by name or MAC address, adjust the command's help accordingly. Signed-off-by:ZhiPeng Lu Reviewed-by: Erik Skultety 2017-11-22 Martin Kletzander build: Fix make dist Since we don't pack symlinks we cannot have recursive loops in them. Since we need one directory to be in tests/vircaps2xmldata/linux-caches/, instead of creating a symlink, just move the files in that directory and adjust tests. 2017-11-21 Michal Privoznik tools: Enable warnings for more binaries/libs Because WARN_CFLAGS and COVERAGE_CFLAGS are not set globally, we rely on each binary built to include WARN_CFLAGS/COVERAGE_CFLAGS. But it is easy to forget those - e.g. libvirt_shell.la. However, don't enable WARN_FLAGS (i.e. don't include AM_CFLAGS) for wireshark plugin - parts of that code are generated and trigger some warnings. Reviewed-by: Erik Skultety 2017-11-21 Michal Privoznik tools: Set CFLAGS for wireshark properly We want to set CFLAGS not CPPFLAGS. Reviewed-by: Erik Skultety 2017-11-21 Michal Privoznik vsh: Make self-test more robust There are couple of limitations when it comes to option types and flags for the options. For instance, VSH_OT_STRING cannot have VSH_OFLAG_REQ set (commit c7543a728). For some reason this is checked in vshCmddefHelp() but not in vshCmddefCheckInternals(). Reviewed-by: Erik Skultety 2017-11-20 Nikolay Shirokovskiy tests: fix typo Reviewed-by: Daniel P. Berrange 2017-11-20 Michal Privoznik qemuBuildDeviceAddressStr: Prefer default alias for PCI bus https://bugzilla.redhat.com/show_bug.cgi?id=1434451 Just like in 9324f67a572f9b32 we need to put default pci-root alias onto the command line instead of the one provided by user. 2017-11-20 Michal Privoznik qemuBuildDriveDevStr: Prefer default alias for SATA bus https://bugzilla.redhat.com/show_bug.cgi?id=1434451 Just like in 9324f67a572f9b32 we need to put default sata alias (which is hardcoded to "ide", obvious, right?) onto the command line instead of the one provided by user. 2017-11-20 Michal Privoznik virQEMUCapsHasPCIMultiBus: Fix @def type This function only queries domain @def. It doesn't change it. Therefore it should take const pointer. 2017-11-20 Andrea Bolognani conf,qemu: Use type-aware switches where possible The compiler can warn us if we add a value to the virDomainChrSerialTargetType enumeration but forget to handle it properly in the code. Let's take advantage of that. This commit is best viewed with 'git diff -w'. Reviewed-by: Pavel Hrdina 2017-11-20 Pino Toscano qemu: add QEMU_CAPS_DEVICE_SCLPLMCONSOLE Add a separate capability for the sclplmconsole device, and check it specifically instead of using QEMU_CAPS_DEVICE_SCLPCONSOLE for that too. Reviewed-by: Andrea Bolognani 2017-11-20 Pino Toscano qemu: rename QEMU_CAPS_SCLP_S390 to QEMU_CAPS_DEVICE_SCLPCONSOLE Give a better name to the capability for the sclpconsole device. Reviewed-by: Andrea Bolognani 2017-11-20 Andrea Bolognani qemu: Add QEMU_CAPS_DEVICE_SPAPR_VTY Up until now we assumed the spapr-vty device would always be present, which is not very nice. Check for its availability before using it instead. Reviewed-by: Pavel Hrdina 2017-11-19 intrigeri AppArmor: add mount rules needed with additional mediation features brought by Linux 4.14 This set of rules was proposed by Christian Boltz on https://bugzilla.opensuse.org/show_bug.cgi?id=1065123. AppArmor: add rules needed with additional mediation features brought by Linux 4.14. 2017-11-18 John Ferlan libvirtd: Fix order of cleanup processing Current cleanup processing is ad-hoc at best - it's led to a couple of strange and hard to diagnose timing problems and crashes. So rather than perform cleanup in a somewhat random order, let's perform cleanup in the exact opposite order of startup. NB: It is possible that virNetlinkEventServerStart fails and we jump to cleanup before driversInitialized has been set. That could leave things inconsistent; however, resolution of that possibility is perhaps more trouble than it's worth to handle. 2017-11-18 John Ferlan libvirtd: Alter order of virNetDaemonNew Let's be sure we can get a Daemon object before the server object. This is a more "orderly" way to do things since the svr object would be added to the dmn object afterwards. libvirtd: Move pid_file_fd setup to before run_dir Once we have forked the daemon successfully, let's claim the pidfile immediately rather than waiting for setup of run_dir. 2017-11-18 Marc-André Lureau news: add vmcoreinfo feature details 2017-11-18 Marc-André Lureau qemu: add vmcoreinfo support Starting from qemu 2.11, the `-device vmcoreinfo` will create a fw_cfg entry for a guest to store dump details, necessary to process kernel dump with KASLR enabled and providing additional kernel details. In essence, it is similar to -fw_cfg name=etc/vmcoreinfo,file=X but in this case it is not backed by a file, but collected by QEMU itself. Since the device is a singleton and shouldn't use additional hardware resources, it is presented as a element in the libvirt domain XML. The device is arm/x86 only for now (targets that support fw_cfg+dma). Related to: https://bugzilla.redhat.com/show_bug.cgi?id=1395248 2017-11-18 Martin Kletzander tests: Add resctrl-skx-twocaches test case to vircaps2xmltest This doesn't add very much value for now, but future test for virresctrl will take information from vircaps2xmldata (since it is dependent on the same info then why duplicate it) and this particular use case helps us cover bit more of the code regarding proper formatting and handling errors. And one more test for vircaps2xmltest doesn't hurt either. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander tests: Minor adjustments for test data This patch modifies some not yet used test data so that the adding a test using this data is a clean patch and not an addition of huge file with some adjustments in small files that will be hidden in the middle of that commit. These changes include: - Add system dir in vircaps2xmldata/linux-caches Back when data for systems with resctrl support were added they had the /sys/fs/system directory put into a system/ subdir of the test and /sys/fs/resctrl in a resctrl/ subdir of that test. However, if we also want a negative test for the resctrl (requesting allocation on a system that does not support resctrl), we need one a test case with any sensible (with cache info) system/ subdir and no resctrl/ one. Easiest way is to add a system -> . symlink into existing test case. - Change default group schemata for linux-resctrl and linux-resctrl-cdp That way we can fit some allocation in. - Remove one cache from resctrl-skx's schemata and make some room for allocations That system already has only one cache, so that file was wrong anyway. We have a version with 2 caches already (linux-resctrl-skx-twocaches), so this will also add variety to future tests. - Add some empty allocation for resctrl-skx Just to have slightly more coverage and variety. We can be sure nothing bad happens if such allocation exists in case we have that in the tests. 2017-11-18 Martin Kletzander tests: Remove executable bits on plain data files Reviewed-by: John Ferlan conf: Format cache banks in capabilities with virFormatIntPretty Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander conf: Sort cache banks in capabilities XML Because the cache banks are initialized based on the order in which their respective directories exist on the filesystem, they can appear in different order. This is here mainly for tests because the cache directory might have different order of children nodes and tests would fail otherwise. It should not be the case with sysfs, but one can never be sure. And this does not take almost any extra time, mainly because it gets initialized once per driver. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Introduce virBitmapShrink Sometimes the size of the bitmap matters and it might not be guessed correctly when parsing from some type of input. For example virBitmapNewData() has Byte granularity, virBitmapNewString() has nibble granularity and so on. virBitmapParseUnlimited() can be tricked into creating huge bitmap that's not needed (e.g.: "0-2,^99999999"). This function provides a way to shrink the bitmap. It is not supposed to free any memory. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Reintroduce virBitmapSubtract Already introduced in the past with 9479642fd3c5, but then renamed to virBitmapIntersect by a908e9e45eb2. This time we'll really use it. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Introduce virBitmapNewString Our bitmaps can be represented as data (raw bytes for which we have virBitmapNewData() and virBitmapToData()), human representation (list of numbers in a string for which we have virBitmapParse() and virBitmapFormat()) and hexadecimal string (for which we have only virBitmapToString()). So let's add the missing complement for the last one so that we can parse hexadecimal strings. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Don't output too many zeros from virBitmapToString Truncate the output so that it is only as big as is needed to fit all the bits, not all the units from the map. This will be needed in the future in order to properly format bitmaps for kernel's sysfs files. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Rename virBitmapDataToString to virBitmapDataFormat It is literally only a wrapper around virBitmapNewData() and virBitmapFormat(), only the naming was wrong since it was introduced. And because we have virBitmap*String functions where the meaning of the 'String' is constant, this might confuse someone. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Rename virBitmapString to virBitmapToString This follows the virBitmapToData() function and, similarly to virBitmapNewData(), we'll be able to have virBitmapNewString() later on without name confusion. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Make prefix optional in virBitampString Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander util: Introduce virFormatIntPretty We can't output better memory sizes if we want to be compatible with libvirt older than the one which introduced /memory/unit, but for new things we can just output nicer capacity to the user if available. And this function enables that. Reviewed-by: John Ferlan 2017-11-18 Martin Kletzander Only output initialized capabilities with VIR_TEST_DEBUG > 1 Currenty virTestInit() outputs all capabilities that it created when running with VIR_TEST_DEBUG=1. Since this is quite a lot of output for every call of this function (and it is not needed until debugging a really deep-down issue) let's just output the info when VIR_TEST_DEBUG is strictly greater than 1. Reviewed-by: Pavel Hrdina 2017-11-18 Martin Kletzander tests: Remove pointless _CFLAGS variables If _CFLAGS for a binary is not specified it uses AM_CFLAGS. So doing $binary_CFLAGS = $(AM_CFLAGS) or $binary_CFLAGS = $(AM_CFLAGS) $(something_that_is_already_in_AM_CFLAGS) is pointless. So remove it for cleaner Makefile.am 2017-11-18 Martin Kletzander build: Use XDR_CFLAGS in more places Since update to glibc-2.26 removed the /usr/include/rpc/rpc.h we used until now, it showed us a problem with not using XDR_CFLAGS properly. On linux that variable has usually -I/usr/include/tirpc because we already probe for it properly, we just don't use it everywhere we need. It is needed by wireshark dissector as well as testutilsqemu.c (through includes) so the build fails with: wireshark/src/packet-libvirt.c:33:10: fatal error: rpc/xdr.h: No such file or directory #include ^~~~~~~~~~~ and In file included from ../src/logging/log_manager.h:29:0, from ../src/qemu/qemu_domain.h:40, from testutilsqemu.c:11: ../src/logging/log_protocol.h:9:10: fatal error: rpc/rpc.h: No such file or directory #include ^~~~~~~~~~~ Since lot of tests use testutilsqemu.c it is easier to add XDR_CFLAGS to AM_CFLAGS than adding it to all $binary_CFLAGS. It's just for tests and we already have bunch of CFLAGS there anyway. 2017-11-17 Jim Fehlig xenconfig: fix compilation error Commit 03d0959a introduced a compilation error in src/xenconfig/xen_xl.c on ARM. Found by Xen's osstest http://logs.test-lab.xenproject.org/osstest/logs/116216/build-armhf-libvirt/6.ts-libvirt-build.log 2017-11-17 Daniel P. Berrange docs: add a page describing support guarantees for libvirt features While we have collective knowledge about the support status of various parts of libvirt, this has never been formally documented, leaving our users to guess. Note, this document makes one change to our previous policy. It explicitly declares the RPC protocol of libvirtd as being a supported interface. THis accepts the reality that we can a) never change it without breaking compat with old libvirt.so, b) there are both rust + go impls that are written against the RPC protocol already. Reviewed-by: John Ferlan Reviewed-by: Jim Fehlig 2017-11-16 Daniel P. Berrange docs: don't use https in XML namespace URIs The XML namespace URI for the QEMU/LXC drivers must use http as the protocol otherwise it won't match the parser's expectations. 2017-11-16 John Ferlan storage: Resolve storage driver crash Resolve a storage driver crash as a result of a long running storageVolCreateXML when the virStorageVolPoolRefreshThread is run as a result of when a storageVolUpload completed and ran the virStoragePoolObjClearVols without checking if the creation code was currently processing a buildVol after incrementing the driver->asyncjob count. The refreshThread will now check the pool asyncjob count before attempting to pursue the pool refresh. Adjust the documentation to describe the condition. Crash from valgrind is as follows (with a bit of editing): ==21309== Invalid read of size 8 ==21309== at 0x153E47AF: storageBackendUpdateVolTargetInfo ==21309== by 0x153E4C30: virStorageBackendUpdateVolInfo ==21309== by 0x153E52DE: virStorageBackendVolRefreshLocal ==21309== by 0x153DE29E: storageVolCreateXML ==21309== by 0x562035B: virStorageVolCreateXML ==21309== by 0x147366: remoteDispatchStorageVolCreateXML ... ==21309== Address 0x2590a720 is 64 bytes inside a block of size 336 free'd ==21309== at 0x4C2F2BB: free ==21309== by 0x54CB9FA: virFree ==21309== by 0x55BC800: virStorageVolDefFree ==21309== by 0x55BF1D8: virStoragePoolObjClearVols ==21309== by 0x153D967E: virStorageVolPoolRefreshThread ... ==21309== Block was alloc'd at ==21309== at 0x4C300A5: calloc ==21309== by 0x54CB483: virAlloc ==21309== by 0x55BDC1F: virStorageVolDefParseXML ==21309== by 0x55BDC1F: virStorageVolDefParseNode ==21309== by 0x55BE5A4: virStorageVolDefParse ==21309== by 0x153DDFF1: storageVolCreateXML ==21309== by 0x562035B: virStorageVolCreateXML ==21309== by 0x147366: remoteDispatchStorageVolCreateXML ... 2017-11-16 Pavel Hrdina qemu: taint domain if virDomainQemuAgentCommand API is used This is similar to the virDomainQemuMonitorCommand API, it can change the domain state in a way that libvirt may not understand. 2017-11-16 Julio Faracco virsh: fixing wrong datatype of 'set-lifcycle-action' command options. The 'set-lifcycle-action' is throwing a weird error after executing it with the '--help' option. The command output is showing the options 'type' and 'action' are as optional, but they aren't. Both are required. virsh # set-lifecycle-action --help ... SYNOPSIS set-lifecycle-action [--type ] [--action ] ... ... OPTIONS [--domain] domain name, id or uuid error: internal error: bad options in command: 'set-lifecycle-action' After applying this patch, both arguments are required now. virsh # set-lifecycle-action --help ... SYNOPSIS set-lifecycle-action [--config] ... Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1509870 2017-11-15 Erik Skultety daemon: virtlockd: Call virNetDaemonGetServer regardless of post exec We need to call it anyway, so the else branch is redundant here. daemon: virtlogd: Drop the server shortcut ref pointer We put the server into a hash table as we do with the other daemons, there is no compelling reason why it should have another pointer dedicated just to the server. Besides, the locking daemon doesn't have it and virtlogd is essentially a copy paste of virtlockd. 2017-11-15 Chen Hanxiao hooks: Fix a wrong description In the definition of virHookQemuOpType and virHookNetworkOpType, we should use 'stopped' rather than 'shutdown'. 2017-11-14 Andrea Bolognani news: Update for configuration of HPT resizing Reviewed-by: John Ferlan docs: Document configuration of HPT resizing Reviewed-by: John Ferlan tests: Add tests for configuration of HPT resizing Reviewed-by: John Ferlan 2017-11-14 Andrea Bolognani qemu: Enable configuration of HPT resizing for pSeries guests Most of the time it's okay to leave this up to negotiation between the guest and the host, but in some situations it can be useful to manually decide the behavior, especially to enforce its availability. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1308743 Reviewed-by: John Ferlan 2017-11-14 Jiri Denemark vircapstest: Avoid (im)possible strcmp call with NULL argument Some compilers may get confused and decide we are calling strcmp with NULL argument from test_virCapsDomainDataLookupLXC. Although this does not really happen since the call is guarded with (data->machinetype != expect_machinetype), using STRNEQ_NULLABLE is easier to understand, less fragile, and doing so makes sure strcmp is never called with NULL argument. 2017-11-14 Jim Fehlig news: add entries for specifying distance between vNUMA cells Add two new entries under new features for 3.10.0. One advertising support for specifying distance between vNUMA cells and another advertising Xen's support for vNUMA configuration. Reviewed-by: Erik Skultety 2017-11-13 Jiri Denemark qemu: Properly report errors from qemuDomainFixupCPUs Reviewed-by: Pavel Hrdina 2017-11-13 Peter Krempa conf: Fix message when maximum vCPU count is less than current Reword the message and drop the numbers (which were reversed) from it so that it actually makes sense. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1509151 2017-11-13 Peter Krempa qemu: process: Setup disk secrets when preparing disks Setup everything related to disks in one place rather than calling in from various places. The change to ordering of the setup steps is necessary since secrets need the master key to be present. 2017-11-13 Peter Krempa qemu: domain: Don't allocate storage source private data if not needed 2017-11-13 Peter Krempa qemu: Tolerate storage source private data being NULL In some cases it does not make sense to pursue that the private data will be allocated (especially when we don't need to put anything in it). Ensure that the code works without it. This also fixes few crashes pointed out in https://bugzilla.redhat.com/show_bug.cgi?id=1510323 2017-11-13 Peter Krempa qemu: parse: Allocate disk definition with private data Use virDomainDiskDefNew instead of VIR_ALLOC in qemuParseCommandLineDisk. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1510781 2017-11-13 Peter Krempa qemu: block: Don't leak server JSON object from protocol generators If creation of the main JSON object containing the storage portion of a virStorageSource would fail but we'd allocate the server structure we'd leak it. Found by coverity. 2017-11-13 Peter Krempa qemu: block: Break out early on invalid storage sources Return NULL right away in qemuBlockStorageSourceGetBackendProps when an invalid storage source is presented so that virJSONValueObjectAdd isn't called with a NULL argument. Found by coverity. 2017-11-13 Peter Krempa conf: Properly parse The terminator would not be parsed properly since the XPath selector was looking for an populated element, and also the code did not bother assigning the terminating virStorageSourcePtr to the backingStore property of the parent. Some tests would catch it if there wasn't bigger fallout from the change to backing store termination in a693fdba0111. Fix them properly now. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1509110 2017-11-13 Michal Privoznik Revert "virNetDevSupportBandwidth: Enable QoS for vhostuser" https://bugzilla.redhat.com/show_bug.cgi?id=1497410 This reverts commit bc8a99ef06417a2303ccab455f9f045e2a617916. The vhostuser is not a TAP. Therefore our QoS code is not able to set any bandwidth. I don't really understand what I was thinking. 2017-11-13 Wim ten Have libxlxml2domconfigtest: add test for vNUMA config Add tests to ensure the libxl_domain_config generator properly handles vNUMA configuration. Reviewed-by: Jim Fehlig 2017-11-13 Wim ten Have xlconfigtest: add tests for vNUMA configuration Add tests for conversion of domXML vNUMA config to/from xen-xl native vNUMA config. Reviewed-by: Jim Fehlig 2017-11-13 Wim ten Have libxl: vnuma support This patch generates a NUMA distance-aware libxl description from the information extracted from a NUMA distance-aware libvirt XML file. By default, if no NUMA node distance information is supplied in the libvirt XML file, this patch uses the distances 10 for local and 20 for remote nodes/sockets. Reviewed-by: Jim Fehlig 2017-11-13 Wim ten Have xenconfig: add domxml conversions for xen-xl This patch converts NUMA configurations between the Xen libxl configuration file format and libvirt's XML format. XML HVM domain on a 4 node (2 cores/socket) configuration: Xen xl.cfg domain configuration: vnuma = [["pnode=0","size=2048","vcpus=0-1","vdistances=10,21,31,21"], ["pnode=1","size=2048","vcpus=2-3","vdistances=21,10,21,31"], ["pnode=2","size=2048","vcpus=4-5","vdistances=31,21,10,21"], ["pnode=3","size=2048","vcpus=6-7","vdistances=21,31,21,10"]] If there is no XML description amongst the data the conversion schema from xml to native will generate 10 for local and 20 for all remote instances. Reviewed-by: Jim Fehlig 2017-11-13 Wim ten Have numa: describe siblings distances within cells Add support for describing NUMA distances in a domain's XML description. Below is an example of a 4 node setup: A defines a NUMA node. describes the NUMA distance from the to the other NUMA nodes (the s). For example, in above XML description, the distance between NUMA node0 and NUMA node2 is 31. Valid distance values are '10 <= value <= 255'. A distance value of 10 represents the distance to the node itself. A distance value of 20 represents the default value for remote nodes but other values are possible depending on the physical topology of the system. When distances are not fully described, any missing sibling distance values will default to 10 for local nodes and 20 for remote nodes. If distance is given for A -> B, then we default B -> A to the same value instead of 20. Reviewed-by: Daniel P. Berrange 2017-11-12 xinhua.Cao remote: Set *neventCallbacks to zero at DEREG_CB To avoid the chance that the loop is run twice, set the neventCallbacks to zero after VIR_FREE(eventCallbacks) was called. 2017-11-10 Pino Toscano tests: fix TTY check in virTestUseTerminalColors() Since colors would be used when writing to stdout, then check that stdout is a TTY, instead of stdin. This avoids the usage of terminal color codes when the output is directed to file. 2017-11-10 Michal Privoznik qemuBuildDriveDevStr: Prefer default aliases for IDE bus https://bugzilla.redhat.com/show_bug.cgi?id=1434451 When testing user aliases it was discovered that for 440fx machine type which has default IDE bus builtin, domain cannot start if IDE controller has the user provided alias. This is because for 440fx we don't put the IDE controller onto the command line (since it is builtin) and therefore any device that is plugged onto the bus must use the default alias. 2017-11-09 Viktor Mihajlovski numa: avoid failure in nodememstats on non-NUMA systems libvirt reports a fake NUMA topology in virConnectGetCapabilities even if built without numactl support. The fake NUMA topology consists of a single cell representing the host's cpu and memory resources. Currently this is the case for ARM and s390[x] RPM builds. A client iterating over NUMA cells obtained via virConnectGetCapabilities and invoking virNodeGetMemoryStats on them will see an internal failure "NUMA isn't available on this host" from virNumaGetMaxNode. An example for such a client is VDSM. Since the intention seems to be that libvirt always reports at least a single cell it is necessary to return "fake" node memory statistics matching the previously reported fake cell in case NUMA isn't supported on the system. 2017-11-09 Dawid Zamirski news: Update for vbox 5.2 support vbox: Add support for 5.2.x Simply add the 5.2 SDK header to the existing unified framework. No other special handling is needed as there's no API break between existing 5.1 and the just added 5.2. vbox: Add vbox 5.2 CAPI header file. Extracted from 5.2 SDK and reindented with cppi 2017-11-09 Jim Fehlig virconf: properly set the end of content There was a recent report of the xen-xl converter not handling config files missing an ending newline https://www.redhat.com/archives/libvir-list/2017-October/msg01353.html Commit 3cc2a9e0 fixed a similar problem when parsing content of a file but missed parsing in-memory content. But AFAICT, the better fix is to properly set the end of the content when initializing the virConfParserCtxt in virConfParse(). This commit reverts the part of 3cc2a9e0 that appends a newline to files missing it, and fixes setting the end of content when initializing virConfParserCtxt. A test is also added to check parsing in-memory content missing an ending newline. Reviewed-by: Daniel P. Berrange 2017-11-09 Peter Krempa conf: Fix type for @liveStatus in virDomainObjListLoadAllConfigs Use bool instead of an int. 2017-11-08 Michal Privoznik qemu-ns: Detect /dev/* mount point duplicates even better In 4f1570720218302 I've tried to make duplicates detection for nested /dev mount better. However, I've missed the obvious case when there are two same mount points. For instance if: # mount --bind /dev/blah /dev/blah # mount --bind /dev/blah /dev/blah Yeah, very unlikely (in qemu driver world) but possible. 2017-11-08 Peter Krempa util: storage: Fix parsing of IPv6 portal address for iSCSI Split on the last colon and avoid parsing port if the split remainder contains the closing square bracket, so that IPv6 addresses are interpreted correctly. 2017-11-08 Andrea Bolognani tests: Add caps for QEMU 2.10.0 on ppc64 Reviewed-by: John Ferlan 2017-11-08 Andrea Bolognani tests: Rename ppc64le caps to ppc64 The architecture itself is called ppc64, and it can run both in big endian and little endian mode - the latter is known as ppc64le. From the (virtual) hardware point of view, ppc64 is a more accurate name so it should be used here. Reviewed-by: John Ferlan 2017-11-08 Andrea Bolognani tests: Add caps for QEMU 2.10.0 on aarch64 (GICv3) Reviewed-by: John Ferlan tests: Add caps for QEMU 2.10.0 on aarch64 (GICv2) Reviewed-by: John Ferlan 2017-11-08 Michal Privoznik news: Document predictable file names for memory-backend-file Reviewed-by: John Ferlan 2017-11-08 Michal Privoznik qemu: Use predictable file names for memory-backend-file In some cases management application needs to allocate memory for qemu upfront and then just let qemu use that. Since we don't want to expose path for memory-backend-file anywhere in the domain XML, we can generate predictable paths. In this case: $memoryBackingDir/libvirt/qemu/$shortName/$alias where $shortName is result of virDomainDefGetShortName(). Reviewed-by: John Ferlan 2017-11-08 Michal Privoznik qemu: Destroy whole memory tree When removing path where huge pages are call virFileDeleteTree instead of plain rmdir(). The reason is that in the near future there's going to be more in the path than just files - some subdirs. Therefore plain rmdir() is not going to be enough. Reviewed-by: John Ferlan 2017-11-08 Michal Privoznik qemu: Rename qemuProcessBuildDestroyHugepagesPath At the same time, move its internals into a separate function so that they can be reused. Reviewed-by: John Ferlan 2017-11-08 Michal Privoznik qemu: Set alias for memory cell in qemuBuildMemoryCellBackendStr Very soon qemuBuildMemoryBackendStr() is going to use memory cell aliases. Therefore set one. At the same time, move it a bit further - if virAsprintf() fails, there's no point in setting rest of the members. Reviewed-by: John Ferlan 2017-11-07 Dawid Zamirski docs: Document autoport behavior in the vbox driver docs: Update vbox driver documentation. * libvirt no longer supports vbox <= 3.x * update XML definition sample to show how to attach disks to VBOX's SAS controller and how to change IDE controller model. * update XML to show how to create RDP display with autoport. 2017-11-07 Jiri Denemark cputest: Add data for Intel(R) Core(TM) i7-7700 CPU Reviewed-by: John Ferlan cputest: Add data for Intel(R) Xeon(R) CPU E5-2650 v4 Reviewed-by: John Ferlan 2017-11-07 Jiri Denemark cputest: Do not drop v[0-9] from CPU names Version is a significant part of some Xeon CPUs. Reviewed-by: John Ferlan 2017-11-07 Dawid Zamirski docs: Update news.xml with vbox changes. vbox: Add SAS controller support In VirtualBox SAS and SCSI are separate controller types whereas libvirt does not make such distinction. This patch adds support for attaching the VBOX SAS controllers by mapping the 'lsisas1068' controller model in libvirt XML to VBOX SAS controller type. If VBOX VM has disks attached to both SCSI and SAS controller libvirt domain XML will have two elements with index and model attributes set accordingly. In this case, each respective element must have
element specified to assign it to respective SCSI controller. vbox: Generate disk address element in dumpxml This patch adds
element to each device since device names alone won't adequately reflect the storage device layout in the VM. With this patch, the ouput produced by dumpxml will faithfully reproduce the storage layout of the VM if used with define. vbox: Process empty removable disks in dumpxml Previously any removable storage device without media attached was omitted from domain XML dump. They're still (rightfully) omitted in snapshot XML dump but need to be accounted properly to for the device names to stay in 'sync' between domain and snapshot XML dumps. 2017-11-07 Dawid Zamirski vbox: Cleanup vboxDumpDisks implementation Primer the code for further changes: * move variable declarations to the top of the function * group together free/release statements * error check and report VBOX API calls used 2017-11-07 Dawid Zamirski vbox: Correctly generate drive name in dumpxml If a VBOX VM has e.g. a SATA and SCSI disk attached, the XML generated by dumpxml used to produce "sda" for both of those disks. This is an invalid domain XML as libvirt does not allow duplicate device names. To address this, keep the running total of disks that will use "sd" prefix for device name and pass it to the vboxGenerateMediumName which no longer tries to "compute" the value based only on current and max port and slot values. After this the vboxGetMaxPortSlotValues is not needed and was deleted. vbox: Swap vboxSnapshotGetReadOnlyDisks arguments So that the function signature matches vboxSnapshotGetReadWriteDisks vbox: Do not free disk definitions on cleanup Both vboxSnapshotGetReadWriteDisks and vboxSnapshotGetReadWriteDisks do not need to free the def->disks on cleanup because it's being done by the caller via virDomainSnaphotDefFree vbox: Cleanup/prepare snasphot dumpxml functions This patch prepares the vboxSnapshotGetReadOnlyDisks and vboxSnapshotGetReadWriteDisks functions for further changes so that the code movement does not obstruct the gist of those future changes. This is done primarily because we'll need to know the type of vbox storage controller as early as possible and make decisions based on that info. vbox: Rename vboxDumpIDEHDDs to vboxDumpDisks Because it deals with other disk types as well not just IDE. Also this function now returns -1 on error vbox: Add vboxDumpStorageControllers vbox: Process element in domain XML With this patch, the vbox driver will no longer attach all supported storage controllers by default even if no disk devices are associated with them. Instead, it will attach only those that are implicitly added by virDomainDefAddImplicitController based on element or if explicitly specified via the element. vbox: Cleanup partially-defined VM on failure Since the VBOX API requires to register an initial VM before proceeding to attach any remaining devices to it, any failure to attach such devices should result in automatic cleanup of the initially registered VM so that the state of VBOX registry remains clean without any leftover "aborted" VMs in it. Failure to cleanup of such partial VMs results in a warning log so that actual define error stays on the top of the error stack. 2017-11-07 John Ferlan storage: Privatize virStoragePoolObj and virStorageVolDefList Move the structures into virstorageobj so that both are known within virstorageobj.c. storage: Use virStoragePoolObjGetDef accessor for new driver events Missed from merge from commit id 'b0652192' into commit id 'bfcd8fc92' were a couple of obj->def-> references. storage: Use virStoragePoolObjGetDef accessor for ZFS backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for VSTORAGE backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for SCSI backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for RBD backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for MPATH backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for iSCSI backend In preparation for privatizing the object, use the accessor. 2017-11-07 Christian Ehrhardt apparmor, virt-aa-helper: allow ipv6 In case ipv6 is used the network inet6 permission is required for virt-aa-helper. 2017-11-07 Christian Ehrhardt apparmor: allow qemu to read max_segments Since qemu 2.9 via 9103f1ce "file-posix: Consider max_segments for BlockLimits.max_transfer" this is a new access that is denied by the qemu profile. It is non fatal, but prevents the fix mentioned to actually work. It should be safe to allow reading from that path. Since qemu opens a symlink path we need to translate that for apparmor from "/sys/dev/block/*/queue/max_segments" to "/sys/devices/**/block/*/queue/max_segments" 2017-11-07 Peter Krempa tests: Add testing of storage backend JSON props formatter Add a new test program called 'qemublocktest' to test the block layer related stuff and test storage source to JSON generator by comparing it to the JSON parser. qemu: block: Add node-names to JSON backing storage strings Format out the node-name if it was assigned for JSON-based storage specification. qemu: block: Add JSON props generator for ssh storage backing qemu: block: Add JSON props generator for sheepdog storage backing qemu: block: Add JSON props generator for RBD storage backing qemu: block: Add JSON props generator for NBD storage backing 2017-11-07 John Ferlan qemu: block: Add JSON props generator for iSCSI protocol 2017-11-07 Peter Krempa qemu: block: Add JSON props generator for 'curl' based storage backends QEMU uses curl for accessing files using http(s) and ftp(s). They share common options so let's generate them in one helper. storage: Store RBD image name as pool and image name Similarly to how we store gluster names, split the name into a pool and image portions when paring the XML and store them separately. 2017-11-07 Peter Krempa storage: Don't store leading '/' in image name when splitting out volume Libvirt historically stores storage source path including the volume as one string in the XML, but that is not really flexible enough when dealing with the fields in the code. Previously we'd store the slash separating the two as part of the image name. This was fine for gluster but it's not necessary and does not scale well when converting other protocols. Don't store the slash as part of the path. The resulting change from absolute to relative path within the gluster driver should be okay, as the root directory is the default when accessing gluster. 2017-11-07 Peter Krempa qemu: process: Split out useful parts from qemuBuildNetworkDriveURI Extract the part formatting the basic URI part so that it can be reused to format JSON backing definitions. Parts specific to the command line format will remain in qemuBuildNetworkDriveURI. The new function is called qemuBlockStorageSourceGetURI. qemu: block: Use proper type for servers for VxHS disks Original implementation used 'SocketAddress' equivalent from qemu for the disk server field, while qemu documentation specifies 'InetSocketAddress'. The backing store parser uses the correct parsing function but the formatter used the incorrect one (and also with the legacy mode enabled which was wrong). qemu: command: Move disk serial validation to qemuCheckDiskConfig qemu: command: Move blkiotune checks to qemuCheckDiskConfig qemu: command: Refactor blkiotune checks to tolerate NULL qemuCaps To allow aggregating the checks, refactor the code to check capabilities only if they were provided. qemu: command: Merge checks from qemuBuildDriveStrValidate to qemuCheckDiskConfig Stash all the disk definition and capability checks into one function. 2017-11-07 Peter Krempa qemu: command: Refactor qemuBuildDriveStrValidate to make qemuCaps optional To allow merging this with other disk type checks we need to check qemuCaps only when available, since some of the checks are executed on disk cold-plug and thus capabilities should not be checked. Make the checks optional by making them conditional on qemuCaps not being NULL. 2017-11-07 Peter Krempa qemu: command: Directly report bus type in qemuBuildDriveStrValidate All of the error message are already in a conditional block with known bus type. Inline the bus type rather than formatting it from a separate variable. qemu: command: Move disk index validation closer to usage The disk index validation is used only in very specific cases and does not need to be performed otherwise. Move it out of the global check into the usage place. qemu: command: Remove dead code when formatting -drive busid and unitid are ever used only if the device is an SD card due to the check in qemuDiskBusNeedsDeviceArg. Since the SD card does not have an bus or unit number, most of the code and command line formatter can be removed since it will never be used. util: Fix condition check in virDiskNameToIndex Use the more common '< 0' rather than the non-zero check. 2017-11-07 Michal Privoznik qemu: Move memPath generation from memoryBackingDir to a separate function In near future we will need more than just a plain VIR_STRDUP(). Better implement that in a separate function and in qemuBuildMemoryBackendStr() which is complicated enough already. Reviewed-by: John Ferlan 2017-11-07 Michal Privoznik conf: s/virDomainObjGetShortName/virDomainDefGetShortName/ This function works over domain definition and not domain object. Its name is thus misleading. Reviewed-by: John Ferlan 2017-11-06 Nikolay Shirokovskiy rpc,lockd: Add missing netserver refcount increment on reload After the virNetDaemonAddServerPostExec call in virtlogd we should have netserver refcount set to 2. One goes to netdaemon servers hashtable and one goes to virt{logd,lock} own reference to netserver. Let's add the missing increment in virNetDaemonAddServerPostExec itself while holding the daemon lock. Since lockd defers management of the @srv object by the presence in the hash table, virLockDaemonNewPostExecRestart must Unref the alloc'd Ref on the @srv object done as part of virNetDaemonAddServerPostExec and virNetServerNewPostExecRestart processing. The virNetDaemonGetServer in lock_daemon main will also take a reference which is Unref'd during main cleanup. 2017-11-06 John Ferlan lockd: Need to Unref @srv when done with it. Commit id '252610f7d' used a hash table to store the @srv, but didn't handle the virObjectUnref if virNetDaemonNew failed nor did it use virObjectUnref once successfully placed into the table which will now be managing it's lifetime (and would cause the virObjectRef if successfully inserted into the table). 2017-11-06 Jiri Denemark conf: Don't inline virDomainNetTypeSharesHostView When coverage build is enabled, gcc complains about it: In file included from qemu/qemu_agent.h:29:0, from qemu/qemu_driver.c:47: qemu/qemu_driver.c: In function 'qemuDomainSetInterfaceParameters': ./conf/domain_conf.h:3397:1: error: inlining failed in call to 'virDomainNetTypeSharesHostView': call is unlikely and code size would grow [-Werror=inline] virDomainNetTypeSharesHostView(const virDomainNetDef *net) ^ 2017-11-04 Michal Privoznik virsh: Define multi line macros properly In some cases there's dangling backward slash at the end of multi line macros. While technically the code works, it will stop if some empty lines are removed. 2017-11-03 Dawid Zamirski domain: Allow 'model' attribute for ide controller The optional values are 'piix3', 'piix4' or 'ich6'. Those will be needed to allow setting IDE controller model in VirtualBox driver. 2017-11-03 Dawid Zamirski vbox: Add more IStorageController API mappings This patch exposes additional methods of the native VBOX API to the libvirt 'unified' vbox API to deal with IStorageController. The exposed methods are: * IStorageController->GetStorageControllerType() * IStorageController->SetStorageControllerType() * IMachine->GetStorageControllers() 2017-11-03 Dawid Zamirski vbox: Support empty removable drives. Original code was checking for non empty disk source before proceeding to actually attach disk device to VM. This prevented from creating empty removable devices like DVD or floppy. Therefore, this patch re-organizes the loop work-flow to allow such configurations as well as makes the code follow better libvirt practices. Additionally, adjusted debug logs to be more helpful - removed old ones and added new which give more valuable info for troubleshooting. vbox: Errors in vboxAttachDrives are now critical Previously, if one tried to define a VBOX VM and the API failed to perform the requested actions for some reason, it would just log the error and move on to process remaining disk definitions. This is not desired as it could result in incorrectly defined VM without the caller even knowing about it. So now all the code paths that call virReportError are now treated as hard failures as they should have been. vbox: Remove unused mediumEmpty Remove the setting since it's unused as of commit 34364df3 which should have never copied it in from the old code which ended up getting removed as part of commit c7c286c6. 2017-11-03 Dawid Zamirski vbox: Cleanup vboxAttachDrives implementation This commit primes vboxAttachDrives for further changes so when they are made, the diff is less noisy: * move variable declarations to the top of the function * add disk variable to replace all the def->disks[i] instances * add cleanup at the end of the loop body, so it's all in one place rather than scattered through the loop body. It's purposefully called 'cleanup' rather than 'skip' or 'continue' because future commit will treat errors as hard-failures. 2017-11-03 Dawid Zamirski vbox: vboxAttachDrives now relies on address info Previously, the driver was computing VBOX's devicePort/deviceSlot values based on device name and max port/slot values. While this worked, it completely ignored
values. Additionally, libvirt's built-in virDomainDiskDefAssignAddress already does a good job setting default values on virDomainDeviceDriveAddress struct which we can use to set devicePort and deviceSlot and accomplish the same result while allowing the customizing those via XML. Also, this allows to remove some code which will make further patches smaller. vbox: Close media when undefining domains When registering a VM we call OpenMedium on each disk image which adds it to vbox's global media registry. Therefore, we should make sure to call Close when unregistering VM so we cleanup the media registry entries after ourselves - this does not remove disk image files. This follows the behaviour of the VBoxManage unregistervm command. vbox: Update ATTRIBUTE_UNUSED usage Since the removal of VBOX <= 3x, the function arguments are actually used so they should not be marked with ATTRIBUTE_UNUSED anymore. 2017-11-03 Andrea Bolognani cfg.mk: Prohibit backspace alignment attempts Now that our codebase is clean, we can add a syntax-check rule to ensure it will remain so. 2017-11-03 Andrea Bolognani Remove backslash alignment attempts Right-aligning backslashes when defining macros or using complex commands in Makefiles looks cute, but as soon as any changes is required to the code you end up with either distractingly broken alignment or unnecessarily big diffs where most of the changes are just pushing all backslashes a few characters to one side. Generated using $ git grep -El '[[:blank:]][[:blank:]]\\$' | \ grep -E '*\.([chx]|am|mk)$$' | \ while read f; do \ sed -Ei 's/[[:blank:]]*[[:blank:]]\\$/ \\/g' "$f"; \ done 2017-11-03 Peter Krempa qemu: domain: skip chain detection to end of backing chain When a user provides the backing chain, we will not need to re-detect all the backing stores again, but should move to the end of the user specified chain. Additionally if a user provides a full terminated chain we should not attempt any further detection. qemu: domain: Prepare TLS data for the whole backing chain Iterate through the backing chain when setting up TLS for disks. qemu: domain: Remove pointless alias check When attaching the disks, aliases are always generated. qemu: domain: Destroy secrets for complete backing chain 2017-11-03 Peter Krempa qemu: domain: Extract setup for disk source secrets Separate it so that it deals only with single virStorageSource, so that it can later be reused for full backing chain support. Two aliases are passed since authentication is more relevant to the 'storage backend' whereas encryption is more relevant to the protocol layer. When using node names, the aliases will be different. 2017-11-03 Peter Krempa qemu: domain: Simplify using DAC permissions of top of backing chain qemuDomainGetImageIds and qemuDomainStorageFileInit are helpful when trying to access a virStorageSource from the qemu driver since they figure out the correct uid and gid for the image. When accessing members of a backing chain the permissions for the top level would be used. To allow using specific permissions per backing chain level but still allow inheritance from the parent of the chain we need to add a new parameter to the image ID APIs. 2017-11-03 Peter Krempa security: selinux: Take parent security label into account Until now we ignored user-provided backing chains and while detecting the code inherited labels of the parent device. With user provided chains we should keep this functionality, so label of the parent image in the backing chain will be applied if an image-specific label is not present. security: dac: Take parent security label into account Until now we ignored user-provided backing chains and while detecting the code inherited labels of the parent device. With user provided chains we should keep this functionality, so label of the parent image in the backing chain will be applied if an image-specific label is not present. security: selinux: Pass parent storage source into image labeling helper virSecuritySELinuxSetImageLabelInternal assigns different labels to backing chain members than to the parent image. This was done via the 'first' flag. Convert it to passing in pointer to the parent virStorageSource. This will allow us to use the parent virStorageSource in further changes. storage: Extract error reporting for broken chains Simplify reporting the error if backing chain is broken for further callers by extracting it into a separate function. storage: Add feature check for storage file backend supporting access check When the user provides backing chain, we don't need the full support for traversing the backing chain. This patch adds a feature check for the virStorageSourceAccess API. storage: Extract common code to retrieve driver backend for support check The 'file access' module of the storage driver has few feature checks to determine whether libvirt supports given storage driver method. The code to retrieve the driver struct needed for the check is the same so it can be extracted. 2017-11-02 Jiri Denemark qemu: Add support for block-incremental migration parameter We handle incremental storage migration in a different way. The support for this new (as of QEMU 2.10) parameter is only needed for full coverage of migration parameters used by QEMU. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Add support for max-bandwidth migration parameter We already support several ways of setting migration bandwidth and this is not adding another one. With this patch we are able to read and write this parameter using query-migrate-parameters and migrate-set-parameters in one call with all other parameters. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Rename TLS related migration parameters The parameters used "migrate" prefix which is pretty redundant and qemuMonitorMigrationParams structure is our internal representation of QEMU migration parameters and it is supposed to use names which match QEMU names. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Add support for setting downtime-limit migration parameter We already support setting the maximum downtime with a dedicated virDomainMigrateSetMaxDowntime API. This patch does not implement another way of setting the downtime by adding a new public migration parameter. It just makes sure any parameter we are able to get from a QEMU monitor by query-migrate-parameters can be passed back to QEMU via migrate-set-parameters. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemumonitorjsontest: Rename 2nd CHECK macro in migration params test The second CHECK macro was used for string parameters. Let's rename it to CHECK_STR and move it up to have all checks in one place. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemumonitorjsontest: Rename 1st CHECK macro in migration params test The first CHECK macro in the test is used for checking integer values. Let's make it a bit more generic to be usable for any numeric type and use it for a new CHECK_INT macro. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Drop giant if statement from qemuMonitorSetMigrationParams The check can be easily replaced with a simple test in the JSON implementation and we don't need to update it every time a new parameter is added. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Use macro for setting string migration parameters Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Generalize APPEND macro in qemuMonitorJSONSetMigrationParams The APPEND macro is now be usable for any type. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Use macro for parsing ull migration parameters Reviewed-by: John Ferlan qemu: Use macro for parsing string migration parameters Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark qemu: Generalize PARSE macro in qemuMonitorJSONGetMigrationParams The macro (now called PARSE_SET) is now usable for any type which needs a *_set bool for indicating a valid value. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark cpu_map: Add cqm alternative name for cmt Linux kernel shows our "cmt" feature as "cqm". Let's mention the name in the cpu_map.xml to make it easier to find. Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark Post-release version bump to 3.10.0 2017-11-02 Daniel Veillard Release of libvirt-3.9.0 * docs/news.xml: update for release * po/*po*: regenerated 2017-11-02 Andrea Bolognani news: Update for 3.9.0 release Reviewed-by: John Ferlan 2017-11-02 Jiri Denemark cputest: Skip tests requiring JSON_MODELS if QEMU is disabled Some tests require JSON_MODELS to be parsed into qemuCaps and applied when computing CPU models and such test cannot succeed if QEMU driver is disabled. Let's mark the tests with JSON_MODELS_REQUIRED and skip the appropriate parts if building without QEMU. On the other hand, CPU tests with JSON_MODELS should succeed even if model definitions from QEMU are not parsed and applied. Let's explicitly test this by repeating the tests without JSON_MODELS set. This fixes the build with QEMU driver disabled, e.g., on some architectures on RHEL/CentOS. Reviewed-by: Pavel Hrdina 2017-10-31 Jiri Denemark spec: Restart libvirtd in posttrans When upgrading libvirt packages, there's no strict ordering for the installation or removal of the individual libvirt sub packages. Thus libvirt-daemon may be upgraded (and its %postun scriptlet) started before all sub packages with driver libraries are upgraded. When libvirt-daemon's %postun scriptlet restarts the daemon old drivers may still be laying around and the daemon may crash when it tries to use them. Let's restart the daemon in %posttrans to make sure libvirtd is restarted only after all sub packages are at the same version. https://bugzilla.redhat.com/show_bug.cgi?id=1464300 2017-10-27 Michal Privoznik virNetDevSupportBandwidth: Enable QoS for vhostuser Since vhostuser type is really a tap that is just plugged into different type of bridge, supporting QoS is trivial. qemuDomainSetInterfaceParameters: Explicitly reject unsupported net types For instance, NET_TYPE_MCAST doesn't support setting QoS. Instead of claiming success and doing nothing, we should be explicit about that and report an error. qemuhotplugtest: Test user supplied alias 2017-10-27 John Ferlan qemu.conf: Clarify the various _tls_x509_cert_dir descriptions https://bugzilla.redhat.com/show_bug.cgi?id=1458630 Apparantly commit id 'dc4c2f75a' wasn't specific enough, so here's a few more clarifications. 2017-10-27 John Ferlan docs: Add news article 2017-10-27 John Ferlan storage: Allow creation of a LUKS using logical volume https://bugzilla.redhat.com/show_bug.cgi?id=1427049 Use virStorageBackendCreateVolUsingQemuImg to apply the LUKS information to the logical volume just created. As part of the processing of the lvcreate command add 2MB to the capacity to account for the LUKS header when it's determined that the volume desires to use encryption. 2017-10-27 John Ferlan storage: Introduce virStorageBackendCreateVolUsingQemuImg Create a shim that will allow other backends to make use of qemu-img functionality to create or possibly modify the volume. storage: Extract out the LVCREATE Refactor to extract out the LVCREATE command. This also removes the need for the local @created since the error path can now only be reached after the creation of the logical volume. docs: Add news article for bug fix 2017-10-27 John Ferlan storage: Properly resize a local volume using LUKS https://bugzilla.redhat.com/show_bug.cgi?id=1490279 Turns out the virStorageBackendVolResizeLocal did not differentiate whether the target volume was a LUKS volume or not and just blindly did the ftruncate() on the target volume. Follow the volume creation logic (in general) and create a qemu-img resize command to resize the target volume for LUKS ensuring that the --object secret is provided as well as the '--image-opts' used by the qemu-img resize logic to describe the path and secret ensuring that it's using the luks driver on the volume of course. 2017-10-27 John Ferlan storage: Alter storageBackendCreateQemuImgSecretObject args Since all that was really needed was a couple of fields and building the object can be more generic, let's alter the args a bit. This will be useful shortly for adding the secret object for a volume resize operation on a luks volume that will need a secret object. storage: Add error path for virStorageBackendCreateQemuImgCmdFromVol Rather than inline the various free's and return NULL, just create an error label. 2017-10-27 John Ferlan storage: Alter args to storageBackendResizeQemuImg Rather than passing just the path, pass the virStorageVolDefPtr as we're going to need it shortly. Also fix the order of code and stack variables in the calling function virStorageBackendVolResizeLocal. 2017-10-27 Christian Ehrhardt virt-aa-helper-test: only fails go to stdout by default By Default (without -d) the tests will only print Failures. So a log should follow general "no message is a good message" style. But the testfw checks always emit the skip info to stdout. Instead they should use the redirection that is controlled by -d. This avoids mesages like the following to clutter the log: Skipping FW AAVMF32 test. Could not find /usr/share/AAVMF/AAVMF32_CODE.fd 2017-10-27 Christian Ehrhardt virt-aa-helper: apparmor wildcards to forbidden chars Some globbing chars in the domain name could be used to break out of apparmor rules, so lets forbid these when in virt-aa-helper. Also adding a test to ensure all those cases were detected as bad char. 2017-10-27 Marc Hartmayer qemu: Fix memory leak in virQEMUCapsLoadCPUModels Don't leak @blockNodes in the loop. ==226576== 7,120 bytes in 60 blocks are definitely lost in loss record 122 of 125 ==226576== at 0x4835214: calloc (vg_replace_malloc.c:711) ==226576== by 0x4950D7B: virAllocN (viralloc.c:191) ==226576== by 0x49EB5BB: virXPathNodeSet (virxml.c:676) ==226576== by 0x104DB67: virQEMUCapsLoadCPUModels (qemu_capabilities.c:3738) ==226576== by 0x105510D: virQEMUCapsLoadCache (qemu_capabilities.c:3929) ==226576== by 0x104459F: qemuTestParseCapabilities (testutilsqemu.c:498) ==226576== by 0x1040DC9: testQemuCapsCopy (qemucapabilitiestest.c:105) ==226576== by 0x1041F07: virTestRun (testutils.c:180) ==226576== by 0x1040B45: mymain (qemucapabilitiestest.c:181) ==226576== by 0x104320F: virTestMain (testutils.c:1119) ==226576== by 0x1041149: main (qemucapabilitiestest.c:193) Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski 2017-10-26 Jim Fehlig daemon: remove minsize directive from hypervisor logrotate files Commit 6c43149c removed the minsize directive from the qemu logrotate file but missed other hypervisors. Remove minsize from the libxl, lxc, and uml logrotate files as well. Reviewed-by: Daniel P. Berrange 2017-10-26 Jim Fehlig qemu: logrotate: drop minsize directive On a cloud host it is possible to create 100's of unique instances per day, each leaving behind a /var/log/libvirt/qemu/instance-name.log file that is < 100k. With the current 'minsize 100k' directive, these files are never rotated and hence never removed. Over months of time, tens of thousands of these files can accumulate on the host. Dropping 'minsize 100k' allows rotating small files, which will increase the number of log files, but 'rotate 4' ensures they will be removed after a month. Reviewed-by: Daniel P. Berrange 2017-10-26 Christian Ehrhardt virt-aa-helper: fix libusb access to udev usb descriptions In bf3a4140 "virt-aa-helper: fix libusb access to udev usb data" the libusb access to properly detect the device/bus ids was fixed. The path /run/udev/data/+usb* contains a subset of that information we already allow to be read and are currently not needed for the function qemu needs libusb for. But on the init of libusb all those files are still read so a lot of apparmor denials can be seen when using usb host devices, like: apparmor="DENIED" operation="open" name="/run/udev/data/+usb:2-1.2:1.0" comm="qemu-system-x86" requested_mask="r" denied_mask="r" Today we could silence the warnings with a deny rule without breaking current use cases. But since the data in there is only a subset of those it can read already it is no additional information exposure. And on the other hand a future udev/libusb/qemu combination might need it so allow the access in the default apparmor profile. 2017-10-26 Christian Ehrhardt virt-aa-helper: grant locking permission on -f Hot-adding disks does not parse the full XML to generate apparmor rules. Instead it uses -f to append a generic rule for that file path. 580cdaa7: "virt-aa-helper: locking disk files for qemu 2.10" implemented the qemu 2.10 requirement to allow locking on disks images that are part of the domain xml. But on attach-device a user will still trigger an apparmor deny by going through virt-aa-helper -f, to fix that add the lock "k" permission to the append file case of virt-aa-helper. 2017-10-26 Jiri Denemark conf: Avoid leaking blockers from virDomainCapsCPUModel When adding CPU usability blockers I forgot to properly free them when in virDomainCapsCPUModelsDispose. Reported-by: Marc Hartmayer Reviewed-by: Pavel Hrdina 2017-10-26 Christian Ehrhardt Increase default file handle limits for virtlockd The assumption so far was an average of 4 disks per guest. But some architectures, like s390x, still often use plenty of smaller disks. To include those in the considerations an assumption of an average of 10 disks is more reasonable. 2017-10-26 Christian Ehrhardt Increase default file handle limits for virtlogd The initial assumption was ~2 files per guest, but some common setups like Openstack drive up to 4 files per guest. E.g. on Arm where the following XML leads to 4 file handles: With that in mind and the target to support 4k guests by default we should raise the limit to 16k. 2017-10-26 Jiri Denemark qemu: Enabled pause-before-switchover migration capability QEMU identified a race condition between the device state serialization and the end of storage migration. Both QEMU and libvirt needs to be updated to fix this. Our migration work flow is modified so that after starting the migration we to wait for QEMU to enter "pre-switchover", "postcopy-active", or "completed" state. Once there, we cancel all block jobs as usual. But if QEMU is in "pre-switchover", we need to resume the migration afterwards and wait again for the real end (either "postcopy-active" or "completed" state). Old QEMU will just enter either "postcopy-active" or "completed" directly, which is still correctly handled even by new libvirt. The "pre-switchover" state will only be entered if QEMU supports it and the pause-before-switchover capability was enabled. Thus all combinations of libvirt and QEMU will work, but only new QEMU with new libvirt will avoid the race condition. 2017-10-26 Jiri Denemark qemu: Add pause-before-switchover migration capability This new capability enables a pause before device state serialization so that we can finish all block jobs without racing with the end of the migration. The pause is indicated by "pre-switchover" state. Once we're done QEMU enters "device" migration state. This patch just defines the new capability and QEMU migration states and their mapping to our job states. 2017-10-26 Jiri Denemark qemu: Add support for migrate-continue QMP command 2017-10-26 Peter Krempa docs: Drop mention that WWN for disks must be unique For multipath disks it might be useful to have the same WWN for multiple disks. It's the users choice to do so. Since we dropped the check that disallows using duplicate WWNs drop the docs as well. https://bugzilla.redhat.com/show_bug.cgi?id=1464975 2017-10-26 Dawid Zamirski vbox: Read runtime RDP port and handle autoport VirutalBox has a IVRDEServerInfo structure available that gives the effective runtime port that the VM is using when it's running. This is useful when the "TCP/Ports" VBox property was set to port range (e.g. via autoport = "yes" or via VBoxManage) in which case it would be impossible to get the "active" port otherwise. 2017-10-25 Dawid Zamirski vbox: Make autoport set RDP port range. Originally autoport in vbox driver was setting the port to default value (3389) which caused multiple VM instances use the same port. Since libvirt XML does not allow to set port ranges, this patch changes the "autoport" behavior to set VBox's "TCP/Ports" property to an arbitrary port range (3389-3689) to avoid that issue. 2017-10-25 Dawid Zamirski vbox: Remove old unflexible macros The VBOX_SESSION_OPEN/CLOSE macros are only called in _vboxDomainSnapshotRestore and they are unflexible because: * assume the caller will have variable named "data" * can only create Write lock type As per above, it's not that hard to simply use the VBOX API directly. 2017-10-24 Peter Krempa virsh: domain: Fix option handling in domxml-to-native Commit fdeac7a05fdf85458d72e89efcfa0f444525aaad tried to fix the output of 'virsh domxml-to-native --help' by switching types around. One of the changes broke the option parser. VSH_OT_ARGV should be used only for variable argument count, not to make the help generator look pretty. The correct option type in this case is VSH_OT_STRING as it's not mandatory now since it can be substituted by using --domain. This makes --help for this command look incorrect, but the parser works as it should. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1494400 2017-10-24 Jiri Denemark qemu: Reset hasManagedSave after removing a corrupted image When starting a domain with managed save image, we try to restore it first. If the image is corrupted, we silently unlink it and just normally start the domain. At this point the domain has no managed save image, yet we did not reset the hasManagedSave flag. https://bugzilla.redhat.com/show_bug.cgi?id=1460962 2017-10-24 Nikolay Shirokovskiy iohelper: use saferead if later write with O_DIRECT One of the usecases of iohelper is to read from pipe and write to file with O_DIRECT. As we read from pipe we can have partial read and then we fail to write this data because output file is open with O_DIRECT and buffer size is not aligned. 2017-10-23 Jiri Denemark qemu: Set correct job status when qemuMigrationRun fails Instead of enumerating all states which need to be turned into QEMU_DOMAIN_JOB_STATUS_FAILED (and failing to add all of them), it's better to mention just the one which needs to be left alone. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Consistently use exit_monitor in qemuMigrationRun Almost every failure in qemuMigrationRun while we are talking to QEMU monitor results in a jump to exit_monitor label. The only exception is removed by this patch. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Don't misuse "ret" in qemuMigrationRun The "ret" variable is used for storing the return value of a function and should not be used as a temporary variable. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Unite error handling in qemuMigrationRun Merge cancel and cancelPostCopy sections with the generic error section, where we can easily decide whether canceling the ongoing migration is required. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Split cleanup and error code in qemuMigrationRun Let cleanup only do things common to both failure and success paths and move error handling code inside the new "error" section. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Refactor qemuMigrationRun a bit Some code which was supposed to be executed only when migration succeeded was buried inside the cleanup code. Reviewed-by: John Ferlan 2017-10-23 Jiri Denemark qemu: Use switch in qemuMigrationCompleted When adding a new job state it's useful to let the compiler complain about places where we need to think about what to do with the new state. Reviewed-by: John Ferlan 2017-10-23 Michal Privoznik news: Document user aliases tests: Test user set aliases for qemu 2017-10-23 Michal Privoznik qemu: Parse alias from inactive XMLs https://bugzilla.redhat.com/show_bug.cgi?id=1434451 This way users can uniquely identify devices at define time. 2017-10-23 Michal Privoznik docs: Document user aliases conf: Format alias even for inactive XMLs We need to format alias even for inactive XMLs since that's the way how users are going to identify their devices. qemuhotplugtest: Load active XML The point of this test is to load live XML and test hotplug. But even though the XMLs we are parsing are live, the parsing is done with VIR_DOMAIN_DEF_PARSE_INACTIVE flag. qemuxml2argvdata: Drop device aliases The qemuxml2argvtest expects the domain XMLs to be inactive ones. Therefore we should pass inactive XMLs. qemuDomainABIStabilityCheck: Check for memory aliases too Since we will be allowing users to set device aliases and memory devices are fragile when it comes to aliases we have to make sure they won't change during migration. Other devices should be fine. conf: Validate user supplied aliases They have to be unique within the domain. As usual, backwards compatibility takes its price. In this particular situation we have a device that is represented twice in a domain and so is its alias. 2017-10-22 Michal Privoznik conf: Parse user supplied aliases If driver that is calling the parse supports user supplied aliases, they can be parsed even for inactive XMLs. However, to avoid any clashes with aliases that libvirt generates, the user ones have to have "ua-" prefix. Note, that some drivers don't have notion of device aliases at all. Also, in order to support user supplied aliases some extra checks need to be done (e.g. during hotplug). Therefore we can't just enable this feature for all the drivers. Thus we need a flag that drivers set to tell parsing code that they can handle user supplied device aliases. 2017-10-22 Michal Privoznik qemu_alias: Be more tolerant if alias don't follow our format When assigning alias to a device we usually iterate over other devices of its kind trying to find next index. We do this by stripping down the prefix and then parsing number at the end, Usually, if the prefix doesn't match the one we are expecting, we just continue with next iteration. Except for couple of functions: qemuGetNextChrDevIndex(), qemuAssignDeviceRedirdevAlias() and qemuAssignDeviceShmemAlias(). 2017-10-20 John Ferlan conf: Rename [n]macs and maxmacs to [n]names and maxnames To avoid further confusion - rename the array elements to what they are. 2017-10-20 Pavel Hrdina conf: duplicate interface name instead of MAC provided to lookup the interface Introduced by 6094d6ec7fc9ea3e28c18c880b76858f06a8b129. Found by running libvirt-perl tests. Reviewed-by: Andrea Bolognani 2017-10-20 Julio Faracco util: Missing 'removeTimeoutImpl' check variable inside virEventRegisterImpl() function. The function virEventRegisterImpl() checks the attempt to replace the registered events. But there is a duplicate variable inside the IF statement. The variable 'removeHandleImpl' was wrongly repeated. One of them needs to be replaced by 'removeTimeoutImpl'. Reviewed-by: Andrea Bolognani 2017-10-20 Jiri Denemark qemu: Enhance debug message in qemuMonitorSetMigrationCapability Reviewed-by: John Ferlan 2017-10-20 Jiri Denemark qemu: Drop qemuMonitorGetMigrationCapability The only remaining user of qemuMonitorGetMigrationCapability is our test suite. Let's replace qemuMonitorGetMigrationCapability with qemuMonitorGetMigrationCapabilities there and drop the unused function. Reviewed-by: John Ferlan 2017-10-20 Jiri Denemark qemu: Use bitmap with migration capabilities All calls to qemuMonitorGetMigrationCapability in QEMU driver are replaced with qemuMigrationCapsGet. Reviewed-by: John Ferlan 2017-10-20 Jiri Denemark qemu: Store supported migration capabilities in a bitmap Each time we need to check whether a given migration capability is supported by QEMU, we call query-migrate-capabilities QMP command and lookup the capability in the returned list. Asking for the list of supported capabilities once when we connect to QEMU and storing the result in a bitmap is much better and we don't need to enter a monitor just to check whether a migration capability is supported. Reviewed-by: John Ferlan 2017-10-20 Jiri Denemark qemu: Create a wrapper around qemuMonitorSetCapabilities The new function is called qemuProcessInitMonitor and it will enter/exit the monitor so that the caller doesn't have to deal with this. The goal of this patch is to simplify the code in qemuConnectMonitor which would otherwise be a bit hairy after the following patch. Reviewed-by: John Ferlan 2017-10-19 John Ferlan interfaces: Convert virInterfaceObjList to virObjectRWLockable Rather than a forward linked list, let's use the virHashTable in order to manage the objsName data. Requires numerous changes from List to Object management similar to many other drivers/vir*obj.c modules 2017-10-19 John Ferlan interface: Make _virInterfaceObjList virObjectRWLockable Modify the allocation to be a real RWLockable object and add the various RWLock{Read|Write} and RWUnlock calls to process the list of interfaces. 2017-10-19 John Ferlan conf: Add/Allow parsing the encryption in the disk source Since the virStorageEncryptionPtr encryption; is a member of _virStorageSource it really should be allowed to be a subelement of the disk for various disk formats: Source{File|Dir|Block|Volume} SourceProtocol{RBD|ISCSI|NBD|Gluster|Simple|HTTP} NB: Simple includes sheepdog, ftp, ftps, tftp That way we can set up to allow the element to be formatted within the disk source, but we still need to be wary from whence the element was read - see keep track and when it comes to format the data, ensure it's written in the correct place. Modify the qemuxml2argvtest to add a parse failure when there is an as a child of *and* an as a child of . The virschematest will read the new test files and validate from a RNG viewpoint things are fine. 2017-10-19 John Ferlan conf: Add/Allow parsing the auth in the disk source Since the virStorageAuthDefPtr auth; is a member of _virStorageSource it really should be allowed to be a subelement of the disk for the RBD and iSCSI prototcols. That way we can set up to allow the element to be formatted within the disk source. Since we've allowed the to be a child of , we'll need to keep track of how it was read so that when writing out we'll know whether to format as child of or . For the argv2xml parsing, let's format under as a preference. Do not allow to be both a child of and . Modify the qemuxml2argvtest to add a parse failure when there is an as a child of *and* an as a child of . Add tests to validate that if the was found in , then the resulting xml2xml and xml2arg works just fine. The two new .args file are exact copies of the non "-source" version of the file. The virschematest will read the new test files and validate from a RNG viewpoint things are fine Update the virstoragefile, virstoragetest, and args2xml file to show the "preference" to place as a child of . 2017-10-19 John Ferlan qemu: Move encinfo from private disk to private disk src Since the encryption information can also be disk source specific move it from qemuDomainDiskPrivate to qemuDomainStorageSourcePrivate Since the last allocated element from qemuDomainDiskPrivate is removed, that means we no longer need qemuDomainDiskPrivateDispose. 2017-10-19 John Ferlan qemu: Relocate qemuDomainSecretInfoPtr to qemuDomainStorageSourcePrivate Since the secret information is really virStorageSource specific piece of data, let's manage the privateData from there instead of at the Disk level. qemu: Introduce privateData object for virStorageSource Add the object definition and helpers to store security-related private data for virStorageSources. util: storage: Introduce privateData for _virStorageSource Introduce the bare necessities to add privateData to _virStorageSource. qemu: Add missing encinfo cleanup When commit id 'da86c6c22' added support for diskPriv->encinfo in qemuDomainSecretDiskPrepare a change to qemuDomainSecretDiskDestroy to was missed. Although qemuDomainDiskPrivateDispose probably would do the trick. 2017-10-19 Michal Privoznik qemu-ns: Detect /dev/* mount point duplicates better https://bugzilla.redhat.com/show_bug.cgi?id=1495511 When creating new /dev for domain ran in namespace we try to preserve all sub-mounts of /dev. Well, not quite all. For instance if /dev/foo/bar and /dev/foo are both mount points, only /dev/foo needs preserving. /dev/foo/bar is preserved with it too. Now, to identify such cases like this one STRPREFIX() is used. That is not good enough. While it works for [/dev/foo/bar; /dev/foo] case, it fails for [/dev/prefix; /dev/prefix2] where the strings share the same prefix but are in fact two different paths. The solution is to use STRSKIP(). Reviewed-by: Erik Skultety 2017-10-19 Michal Privoznik conf: Pass xmlopt down to virDomainDeviceInfoParseXML This function is going to make decisions based on the features set per each driver. For that we need the virDomainXMLOption object. qemu: Be tolerant to preexisting aliases In the future, some aliases might be already parsed therefore we should avoid overwriting them. qemu: Move device alias assignment to separate functions Let's move all the virAsprintf()-s into separate functions for better structure of the code. Later, when somebody wants to generate a device alias, all they need is to expose the function. qemuAssignDeviceAliases: Use qemuAssignDeviceRNGAlias for assigning RNG aliases We have a special function for assigning aliases to RNG devices. Use that instead of plain virAsprintf(). virDomainObjGetOneDefState: Fix error message It looks like the error message was copied from virsh, because that's where we have @ctl. Nevertheless, it's @flags which is invalid, not @ctl. 2017-10-19 Peter Krempa qemu: hotplug: Reuse qemuDomainAttachDiskGeneric in qemuDomainAttachUSBMassStorageDevice Apart from killing a lot of code this also "implements" authentication and encryption for USB disks. qemu: hotplug: Reuse qemuDomainAttachDiskGeneric in qemuDomainAttachSCSIDisk Get rid of the first copy of the mess. qemu: hotplug: extract disk hotplug worker code This horrible piece of spaghetti code is copy-past(ae)d in the SCSI and USB disk hotplug code with minimal changes. Extract it for further reuse. qemu: hotplug: Use disk target in debug/warning messages where appropriate Some messages deal with the disk itself thus using the disk target is better than using the disk source name which can be NULL in some cases. qemu: hotplug: Remove wrong check for empty disks The check if the disk is empty is wrong and would spuriously reject NBD sources. Remove it. qemu: address: Remove dead code when un-reserving PCI address The code can't fail so having error handling is pointless. 2017-10-19 Ján Tomko fix error message spacing in qemuDomainDetachNetDevice Move the space after the colon. 2017-10-19 Ján Tomko qemu: remove pointless address validation on hot unplug Back in the times of using 'pci_del', unplugging a device without a PCI address was not wired up. After completely removing support for qemu without QEMU_CAPS_DEVICE, aliases are used to uniquely identify devices in all cases. Remove the pointless validation of data that was already present in the domain definition. 2017-10-19 Ján Tomko qemuIsMultiFunctionDevice: return early for non-PCI addresses There is no point in iterating over all devices if none of them could possibly match. 2017-10-19 Ján Tomko qemu: implement input device hotunplug Allow unplugging USB and virtio USB devices. https://bugzilla.redhat.com/show_bug.cgi?id=1379603 2017-10-19 Ján Tomko qemu: implement input device hotplug For both virtio input devices and USB input devices. https://bugzilla.redhat.com/show_bug.cgi?id=1379603 2017-10-19 Ján Tomko Introduce qemuBuildInputDevStr A function that builds the -device string for input devices. split out qemuAssignDeviceInputAlias Move assignment of input device alias into a separate function, for reuse on hotplug. qemu: allow cold unplugging of input devices https://bugzilla.redhat.com/show_bug.cgi?id=1379603 qemu: allow coldplugging input devices https://bugzilla.redhat.com/show_bug.cgi?id=1379603 2017-10-19 Ján Tomko Use qemuDomainEnsureVirtioAddress where possible There are two more cases where we set an S390/CCW/PCI address type based on the machine type. Reuse qemuDomainEnsureVirtioAddress to reduce repetition. 2017-10-19 Ján Tomko Split out qemuDomainEnsureVirtioAddress Split out the common code responsible for reserving/assigning PCI/CCW addresses for virtio disks into a helper function for reuse by other virtio devices. Move qemuCheckCCWS390AddressSupport to qemu_domain Let it be reused in qemu_domain_address. 2017-10-19 Ján Tomko qemuDomainAttachRNGDevice: do not access source.file randomly We pass the source.file to qemuCheckCCWS390AddressSupport for the purpose of reporting an error message without actually checking that the rng device is of type VIR_DOMAIN_RNG_BACKEND_RANDOM. Change it to a hardcoded "rng" string, which also avoids referring to the device by a host-side attribute. 2017-10-19 Ján Tomko qemuDomainAttachControllerDevice: remove dead code After a successful attach, the device address has already been set. Remove the pointless assignment. conf: audit passthrough input devices at domain startup Introduce virDomainAuditInput and use it to log the evdev passed to the guest. 2017-10-19 Pavel Hrdina qemu: implement virDomainSetLifecycleAction() API There is one limitation for using this API, when the guest is started with all actions set to "destroy" we put "-no-reboot" on the QEMU command line. That cannot be changed while QEMU is running and the QEMU process is always terminated no matter what is configured for any action. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1460677 Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina qemu: send allowReboot in migration cookie We need to send allowReboot in the migration cookie to ensure the same behavior of the virDomainSetLifecycleAction() API on the destination. Consider this scenario: 1. On the source the domain is started with: destroy restart destroy 2. User calls an API to set "destroy" for : destroy destroy destroy 3. The guest is migrated to a different host 4a. Without the allowReboot in the migration cookie the QEMU process on destination would be started with -no-reboot which would prevent using the virDomainSetLifecycleAction() API for the rest of the guest lifetime. 4b. With the allowReboot in the migration cookie the QEMU process on destination is started without -no-reboot like it was started on the source host and the virDomainSetLifecycleAction() API continues to work. The following patch adds a QEMU implementation of the virDomainSetLifecycleAction() API and that implementation disallows using the API if all actions are set to "destroy" because we add "-no-reboot" on the QEMU command line. Changing the lifecycle action is in this case pointless because the QEMU process is always terminated. Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina qemu: move detection whether to use -no-reboot to qemu_domain This will be used later on in implementation of new API virDomainSetLifecycleAction(). In order to use it, we need to store the value in status XML to not lose the information if libvirtd is restarted. If some guest was started by old libvirt where it was not possible to change the lifecycle action for running guest, we can safely detect it based on the current actions from the status XML. Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina virsh: introduce set-lifecycle-action command Reviewed-by: John Ferlan lib: introduce virDomainSetLifecycleAction() API Reviewed-by: John Ferlan qemu: pass priv data instead of qemuCaps and autoNodeset Reviewed-by: John Ferlan qemu: pass priv data to qemuBuildMasterKeyCommandLine Reviewed-by: John Ferlan qemu: pass priv data to qemuBuildPMCommandLine Reviewed-by: John Ferlan qemu: pass priv data to qemuBuildMonitorCommandLine Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina qemu: pass virDomainObjPtr to qemuBuildCommandLine Extract the required data inside a function instead of passing it all as arguments. Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina conf: merge virDomainLifecycleCrashAction with virDomainLifecycleAction There is no need to have two different enums where one has the same values as the other one with some additions. Currently for on_poweroff and on_reboot we allow only subset of actions that are allowed for on_crash. This was covered in parse time using two different enums. Now to make sure that we don't allow setting actions that are not supported we need to check it while validating domain config. Reviewed-by: John Ferlan 2017-10-19 Pavel Hrdina conf: introduce virDomainLifecycle enum to list all lifecycle types Reviewed-by: John Ferlan conf: rename virDomainLifecycleAction enum functions Reviewed-by: John Ferlan conf: rename lifecycle enum values to correspond with typedef keyword Reviewed-by: John Ferlan 2017-10-19 Jiri Denemark bhyve: Fix build Commit v3.8.0-95-gfd885a06a dropped nmodels parameter from several APIs in src/cpu/cpu.h, but failed to update all callers. 2017-10-19 Erik Skultety nodedev: udev: Hook up virFileWaitForAccess to work around uevent race If we find ourselves in the situation that the 'add' uevent has been fired earlier than the sysfs tree for a device was created, we should use the best-effort approach and give kernel some predetermined amount of time, thus waiting for the attributes to be ready rather than discarding the device from our device list forever. If those don't appear in the given time frame, we need to move on, since libvirt can't wait indefinitely. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1463285 2017-10-19 Erik Skultety util: Introduce virFileWaitForExists Since we have a number of places where we workaround timing issues with devices, attributes (files in general) not being available at the time of processing them by calling usleep in a loop for a fixed number of tries, we could as well have a utility function that would do that. Therefore we won't have to duplicate this ugly workaround even more. nodedev: udev: Convert udevEventHandleThread to an actual thread routine Adjust udevEventHandleThread to be a proper thread routine running in an infinite loop handling devices. The handler thread pulls all available data from the udev monitor and only then waits until a wakeup signal for new incoming data has been emitted by udevEventHandleCallback. nodedev: udev: Split udevEventHandleCallback in two functions This patch splits udevEventHandleCallback in two (introduces udevEventHandleThread) in order to be later able to refactor the latter to actually become a normal thread which will wait some time for the kernel to create the whole sysfs tree for a device as we cannot do that in the event loop directly. nodedev: udev: Unlock the private data before setting up 'system' node udevSetupSystemDev only needs the udev data lock to be locked because of calling udevGetDMIData which accesses some protected structure members, but it can do that on its own just fine, no need to hold the lock the whole time. nodedev: udev: Remove driver locks from stateInitialize and stateCleanup The driver locks are unnecessary here, since currently the cleanup is only called from the main daemon thread, so we can't race here. Moreover @devs and @privateData are self-lockable objects, so no problem there either. nodedev: udev: Convert udev private data to a lockable object Since there's going to be a worker thread which needs to have some data protected by a lock, the whole code would just simply get unnecessary complex, since two sets of locks would be necessary, driver lock (for udev monitor and event handle) and a mutex protecting thread-local data. Given the future thread will need to access the udev monitor socket as well, why not protect everything with a single lock, even better, by converting the driver's private data to a lockable object, we get the automatic object disposal feature for free. nodedev: udev: Introduce udevEventMonitorSanityCheck helper function We need to perform a sanity check on the udev monitor before every use so that we know nothing has changed in the meantime. The reason for moving the code to a separate helper is to enhance readability and shift the focus on the important stuff within the udevEventHandleCallback handler. nodedev: Move privileged flag from udev private data to driver's state Even though hal doesn't make use of it, the privileged flag is related to the daemon/driver rather than the backend actually used. While at it, get rid of some tab indentation in the driver state struct. 2017-10-18 Chen Hanxiao virsh: domifaddr: clarify description of --full option Option --full will always display the name and MAC address of the the interface. Both virsh help and virsh man page didn't mention that. 2017-10-18 Erik Skultety maint: Remove not-so-much informative block commentaries There were a bunch of commentary blocks that were literally useless in terms of describing what the code following them does, since most of them were documenting "the obvious" or it just wouldn't help at all. maint: Replace tabs with spaces in all source files in repo So we have a syntax-check rule to catch all tab indents but it naturally can't catch tab spacing, i.e. as a delimiter. This patch is a result of running 'vim -en +retab +wq' (using tabstop=8 softtabstop=4 shiftwidth=4 expandtab) on each file from a list generated by the following: find . -regextype gnu-awk \ -regex ".*\.(rng|syms|html|s?[ch]|py|pl|php(\.code)?)(\.in)?" \ | xargs git grep -lP "\t" 2017-10-18 Peter Krempa util: storagefile: Track whether a virStorageSource was auto-detected When formatting an inactive or migratable XML we will need to suppress backing chain members which were detected from the disk to keep semantics straight. This means we need to record, whether a virStorageSource originates from autodetection. qemu: block: Add support for file/block/dir storage to JSON disk src generator qemuBlockStorageSourceGetBackendProps now is able to format the JSON definition for regular storage too. qemu: command: Separate wrapping of disk backend props to 'file' object The file object is needed when formatting the command line, but it makes nesting of the objects less easy for use with blockdev. Separate the wrapping into the 'file' object into a helper used specifically for disk sources in the old code path. 2017-10-18 Kothapally Madhu Pavan qemu: Move qemuFreeKeywords into qemu_parse_command.c Move qemuFreeKeywords into qemu_parse_command.c as qemuParseKeywordsFree and call it rather than inline code in multiple places. 2017-10-18 Dawid Zamirski docs: Fix multiUser/replaceUser in RDP display doc. The original description from commit id '24e0171b' got it backwards. 2017-10-17 Jiri Denemark cputest: Add query-cpu-definitions reply for Xeon-E7-8890 Reviewed-by: John Ferlan 2017-10-17 Jiri Denemark cputest: Update Xeon-E7-8890 data Without the fix in the previous patch the JSON data from QEMU would be interpreted as Haswell-noTSX because x86DataFilterTSX would filter rtm and hle features as a result of family == 6 && model == 63 && stepping < 4 test even though this CPU has stepping == 4. Reviewed-by: John Ferlan 2017-10-17 Jiri Denemark qemu: Parse CPU stepping from query-cpu-model-expansion Even though only family and model are used for matching CPUID data with CPU models from cpu_map.xml, stepping is used by x86DataFilterTSX which is supposed to disable TSX on CPU models with broken TSX support. Thus we need to start parsing stepping from QEMU to make sure we don't disable TSX on CPUs which provide working TSX implementation. See the following patch for a real world example of such CPU. Reviewed-by: John Ferlan 2017-10-17 Nikolay Shirokovskiy conf: fix use of uninitialized variable If same boot order is specified twice (or more) in domain xml we call free for uninitiaziled loadparm on cleanup in virDomainDeviceBootParseXML and SIGABRT (or similar) as a result. vz: fix typo for 0d3d020b vz: missing pieces for fd885a06 for vz driver 2017-10-17 Jiri Denemark qemu: Fix CPU model broken by older libvirt When libvirt older than 3.9.0 reconnected to a running domain started by old libvirt it could have messed up the expansion of host-model by adding features QEMU does not support (such as cmt). Thus whenever we reconnect to a running domain, revert to an active snapshot, or restore a saved domain we need to check the guest CPU model and remove the CPU features unknown to QEMU. We can do this because we know the domain was successfully started, which means the CPU did not contain the features when libvirt started the domain. https://bugzilla.redhat.com/show_bug.cgi?id=1495171 Reviewed-by: Pavel Hrdina 2017-10-17 Jiri Denemark qemu: Filter CPU features when using host CPU When reconnecting to a domain started with a host-model CPU which was started by old libvirt that did not replace host-model with the real CPU definition, libvirt replaces the host-model CPU with the CPU from capabilities (because this is what the old libvirt did when it started the domain). Without this patch libvirt could use features unknown to QEMU in the CPU definition which replaced the original host-model CPU. Such domain would keep running just fine, but any attempt to migrate it will fail and once the domain is saved or snapshotted, restoring it would fail too. In other words whenever we want to use the CPU definition from host capabilities as a guest CPU definition, we have to filter the unknown features. https://bugzilla.redhat.com/show_bug.cgi?id=1495171 Reviewed-by: Pavel Hrdina 2017-10-17 Jiri Denemark conf: Introduce virCPUDefFindFeature Reviewed-by: Pavel Hrdina 2017-10-17 Jiri Denemark qemu: Separate CPU updating code from qemuProcessReconnect The new function is called qemuProcessRefreshCPU. Reviewed-by: Pavel Hrdina 2017-10-17 Jiri Denemark qemu: Check QEMU error on failed migration When migration fails, QEMU may provide a description of the error in the reply to query-migrate QMP command. We can fetch this error and use it instead of the generic "unexpectedly failed" message. Reviewed-by: Pavel Hrdina 2017-10-17 Ján Tomko docs: remove duplicate https links Commit e371b3b changed all the links to libvirt.org to use https. Remove the leftover 'http' links from downloads page, since they point to https anyway. 2017-10-17 Peter Krempa Terminate backing chains explicitly Express a properly terminated backing chain by putting a virStorageSource of type VIR_STORAGE_TYPE_NONE in the chain. The newly used helpers simplify this greatly. The change fixes a bug as formatting an incomplete backing chain and parsing it back would end up in expressing a terminated chain since src->backingStoreRaw was not populated. By relying on the terminator object this can be now processed appropriately. 2017-10-17 Peter Krempa util: storagefile: Add helpers to check presence of backing store Add helpers that will simplify checking if a backing file is valid or whether it has backing store. The helper virStorageSourceIsBacking returns true if the given virStorageSource is a valid backing store member. virStorageSourceHasBacking returns true if the virStorageSource has a backing store child. Adding these functions creates a central points for further refactors. 2017-10-17 Peter Krempa test: set 'type' field of virStorageSource Set the type so that the iterators will work after upcoming modification. storage: Fill in 'type' field for virStorageSource in storage driver Storage driver uses virStorageSource only partially to store it's configuration but fully when parsing backing files of storage volumes. This patch sets the 'type' field to a value other than VIR_STORAGE_TYPE_NONE so that further patches can add a terminator element to backing chains without breaking iteration. util: storagefile: Tolerate NULL path when looking up volume in chain chain->path may be NULL e.g. for NBD drives, so the check needs to avoid dereferencing the path in such case util: storage: use stored index to lookup disks We can now use the backing store ID directly rather than counting the number of images seen while looking up images. 2017-10-17 Peter Krempa util: storage: Store backing chain index in virStorageSource The backing store indexes were not bound to the storage sources in any way. To allow us to bind a given alias to a given storage source we need to save the index in virStorageSource. The backing store ids are now generated when detecting the backing chain. Since we don't re-detect the backing chain after snapshots, the numbering needs to be fixed there. 2017-10-17 Peter Krempa conf: Make backing store index optional Index will remain an internal property even if we allow backing store parsing from the XML, so we need to allow backing store without it in the schema. conf: domain: Simplify return from backing store parser Use VIR_STEAL_PTR to remove conditional cleanup. 2017-10-16 Kothapally Madhu Pavan qemu: argv: parse qemu commandline memory arguments Existing qemuParseCommandLineMem() will parse "-m 4G" format string. This patch allows it to parse "-m size=8126464k,slots=32,maxmem=33554432k" format along with existing format. And adds a testcase to validate the changes. 2017-10-16 Ladi Prosek hyperv: Map Limit to max_memory and VirtualQuantity to cur_balloon Hyper-V uses its own specific memory management so no mapping is going to be perfect. However, it is more correct to map Limit to max_memory (it really is the upper limit of what the VM may potentially use) and keep cur_balloon equal to total_memory. The typical value returned from Hyper-V in Limit is 1 TiB, which is not really going to work if interpreted as "startup memory" to be ballooned away later. 2017-10-16 Ladi Prosek hyperv: Escape WQL queries The code was vulnerable to SQL injection. Likely not a security issue due to WMI SQL and other constraints but still lame. For example: virsh # dominfo \" error: failed to get domain '"' error: internal error: SOAP fault during enumeration: code 's:Sender', subcode 'n:CannotProcessFilter', reason 'The data source could not process the filter. The filter might be missing or it might be invalid. Change the filter and try the request again. ', detail 'The WS-Management service cannot process the request. The WQL query is invalid. ' This commit fixes the Hyper-V driver by escaping all WMI SQL string parameters. The same command with the fix: virsh # dominfo \" error: failed to get domain '"' error: Domain not found: No domain with name " 2017-10-16 Ladi Prosek hyperv: Fix hypervInitConnection error reporting "%s is not a Hyper-V server" is not a correct generalization of all possible error conditions of hypervEnumAndPull. For example: $ virsh --connect hyperv://localhost/?transport=http Enter username for localhost [administrator]: Enter administrator's password for localhost: error: failed to connect to the hypervisor error: internal error: localhost is not a Hyper-V server This commit removes the general virReportError from hypervInitConnection and also the "Invalid query" virReportError from hypervSerializeEprParam, which does not correctly describe the error either (virBufferCheckError has already set a meaningful error message at that point). The same scenario with the fix: $ virsh --connect hyperv://localhost/?transport=http Enter username for localhost [administrator]: Enter administrator's password for localhost: error: failed to connect to the hypervisor error: internal error: Transport error during enumeration: User, password or similar was not accepted (26) 2017-10-16 Daniel P. Berrange qemu: ensure TLS clients always verify the server certificate The default_tls_x509_verify (and related) parameters in qemu.conf control whether the QEMU TLS servers request & verify certificates from clients. This works as a simple access control system for servers by requiring the CA to issue certs to permitted clients. This use of client certificates is disabled by default, since it requires extra work to issue client certificates. Unfortunately the code was using this configuration parameter when setting up both TLS clients and servers in QEMU. The result was that TLS clients for character devices and disk devices had verification turned off, meaning they would ignore errors while validating the server certificate. This allows for trivial MITM attacks between client and server, as any certificate returned by the attacker will be accepted by the client. This is assigned CVE-2017-1000256 / LSN-2017-0002 Reviewed-by: Eric Blake 2017-10-16 Daniel P. Berrange Use https:// links for most sites This adds a rule to require https links for the libvirt, qemu and kvm websites. Remove obsolete TODO file We stopped autogenerating a TODO file from RFE bugs a long time ago. 2017-10-16 Ján Tomko qemu: reserve PCI addresses for implicit i440fx devices Somewhere around commit 9ff9d9f reserving entire PCI slots was eliminated, as demonstrated by commit 6cc2014. Reserve the functions required by the implicit devices: 00:01.0 ISA Bridge 00:01.1 IDE Controller 00:01.2 USB Controller (unless USB is disabled) 00:01.3 Bridge https://bugzilla.redhat.com/show_bug.cgi?id=1460143 2017-10-16 Jiri Denemark cputest: Make a crippled version of Core-i7-2600 xsaveopt is artificially removed from the host to test disabled feature which is only included in QEMU's version of the CPU model. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Update Core-i7-2600 data arat is now enabled even if the hardware does not support it. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Add query-cpu-definitions reply for Xeon-E3-1245 Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Update Xeon-E3-1245 data xsaves is supported by current QEMU/KVM on this CPU. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Add query-cpu-definitions reply for Xeon-E7-4830 This CPU was incorrectly detected as SandyBridge before because the number of additional elements was the same for both SandyBridge and Westmere CPU models, but SandyBridge is newer (the CPU signature does not help here because it doesn't match any signature defined in cpu_map.xml). But since QEMU's version of SandyBridge CPU model contains xsaveopt which needs to be disabled, Westmere becomes the best CPU model when translating CPUID data to virCPUDef. Unfortunately, this doesn't help with translating the data we got from QEMU and the CPU model is still computed as SandyBridge in this case. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Add CPUID data for Intel(R) Xeon(R) CPU E7-4830 Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Add query-cpu-definitions reply for Core-i5-2540M The unavailable features do not make any difference in this case, because this is a SandyBridge CPU which has an empty list of unavailable features. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Use CPU models from QEMU when available When testing cpuDecode for computing guest CPU definition from CPUID data (the CPU definition reported by domain capabilities), we need to use CPU models (and their usability blockers) from QEMU if they are available to cpuDecode in the same way it is actually used in the qemu driver. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Separate QEMUCaps creation from cpuTestCPUIDJson To make the code reusable by other tests. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Test CPU usability blockers Gather query-cpu-definitions results and use them for testing CPU model usability blockers in CPUID to virCPUDef translation. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Print correct feature in virCPUUpdateLive test If the actual result does not match our expectation, the tests would not correctly show the difference if a CPU feature is disabled in the expected result and the actual result does not mention it at all. The test could complain about an unrelated CPU feature or it could even crash in case the actual result contains no more features to go through. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Avoid calling json_reformat in cpu-parse.sh Various version of json_reformat use different number of spaces for indenting. Let's use a simple python reformatter to gain full control over the formatting for consistent results. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cputest: Replace bool with cpuTestCPUIDJson enum We will soon need to handle more than two values. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cpu_x86: Disable blockers from unusable CPU models When decoding CPUID data to virCPUDef we need to be careful about using a CPU model which cannot be directly used on the current host. Normally, libvirt would notice the features which prevent the model from being usable and it would disable them in the computed virCPUDef, but this won't work in case the definition of the CPU model in QEMU contains more features than what we have in cpu_map.xml. We need to count with the usability blockers we got from QEMU and explicitly disable all of them to make the computed virCPUDef usable. https://bugzilla.redhat.com/show_bug.cgi?id=1464832 Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cpu_x86: Move x86FeatureFind* to avoid forward prototypes Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark conf: Introduce virDomainCapsCPUModelsGet This internal API can be used to find a specific CPU model in virDomainCapsCPUModels list. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cpu: Drop unused parameter from cpuDecode The "preferred" parameter is not used by any caller of cpuDecode anymore. It's only used internally in cpu_x86 to implement cpuBaseline. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark cpu: Use virDomainCapsCPUModelsPtr in cpu driver APIs All APIs which expect a list of CPU models supported by hypervisors were switched from char **models and int models to just accept a pointer to virDomainCapsCPUModels object stored in domain capabilities. This avoids the need to transform virDomainCapsCPUModelsPtr into a NULL-terminated list of model names and also allows the various cpu driver APIs to access additional details (such as its usability) about each CPU model. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark qemu: Parse unavailable features for CPU models query-cpu-definitions QMP command returns a list of unavailable features which prevent CPU models from being usable on the current host. So far we only checked whether the list was empty to mark CPU models as (un)usable. This patch parses all unavailable features for each CPU model and stores them in virDomainCapsCPUModel as a list of usability blockers. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark qemu: Store CPU usability blockers in caps cache Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark conf: Add usability blockers to virDomainCapsCPUModel When a hypervisor marks a CPU model as unusable on the current host, it may also give us a list of features which prevent the model from being usable. Storing this list in virDomainCapsCPUModel will help the CPU driver with creating a host-model CPU configuration. Reviewed-by: John Ferlan 2017-10-16 Jiri Denemark util: Introduce virStringListCopy The API makes a deep copy of a NULL-terminated string list. Reviewed-by: John Ferlan 2017-10-14 Michal Privoznik virDomainDeviceInfoParseXML: Clear whole @info on failure Currently, if parsing of device info fails info->alias is freed. It doesn't make much sense to leave the rest of the struct behind. virDomainDeviceInfoParseXML: Separate address parsing into separate func There's one 'return' in the middle of the function body. It's very easy to miss and so it makes adding new code harder. Also the function doesn't follow our style 100%. 2017-10-14 Michal Privoznik virDomainInterfaceStats: Accept MAC addresses properly https://bugzilla.redhat.com/show_bug.cgi?id=1497396 In 0d3d020ba6c4f I've added capability to accept MAC addresses for the API too. However, the implementation was faulty. It needs to lookup the corresponding interface in the domain definition and pass the ifname instead of MAC address. Reviewed-by: John Ferlan 2017-10-14 John Ferlan nodedev: Fix missing network devices Commit id '8708ca01c' added a check to determine whether the NIC had Switchdev capabilities; however, in doing so inadvertently would cause network devices without a PCI device to not be added to the node device database. Thus, network devices having a "computer" as a parent, such as "net_lo*", "net_virbr*", "net_tun*", "net_vnet*", etc. were not added. Alter the check to not even check for Switchdev bits if no PCI device found. 2017-10-13 Michal Privoznik virsh: Deal with multiple matching devices in domif-getlink The command tries to match interface in domain definition by MAC address or interface name. However, since it's possible to configure two interfaces with the same MAC address, it may happen that the XPath returns two or more nodes. We should check for that. Reviewed-by: John Ferlan 2017-10-13 Michal Privoznik virDomainInterfaceStats: Accept MAC address too https://bugzilla.redhat.com/show_bug.cgi?id=1497396 The other APIs accept both, ifname and MAC address. There's no reason virDomainInterfaceStats can't do the same. Reviewed-by: John Ferlan 2017-10-13 Michal Privoznik virDomainNetFind: Report error if no device found Every caller reports the error themselves. Might as well move it into the function and thus unify it. Reviewed-by: John Ferlan 2017-10-13 Michal Privoznik virsh: Document limitation of domifstat https://bugzilla.redhat.com/show_bug.cgi?id=1497396 The current implementation reads the stats from the host. However, this doesn't work for all types of interfaces as not all of them have a representation in the host. For instance, interface type='user' doesn't. Reviewed-by: John Ferlan 2017-10-13 Daniel P. Berrange Add CII best practices badge 2017-10-13 John Ferlan nodedev: Convert virNodeDeviceObjHasCap to bool It only returns 0 or 1 anyway, let's be realistic nodedev: Convert virNodeDeviceObjList to use RWObjectLockable Let's use the RWObjectLockable for the various list lock mgmt. Only time need Write lock will be for Add/Remove logic. secrets: Convert to use ObjectRWLockable Let's use the ObjectRWLockable for the various list lock mgmt. Only time need Write lock will be for Add/Remove logic. network: Convert virNetworkObjList to use RWObjectLockable Let's use the RWObjectLockable for the various list lock mgmt. Only time need Write lock will be for Add, Remove, and Prune logic. 2017-10-13 John Ferlan util: Resolve resource leak Need to free @groups in the parent on success similar to other APIs (virFile*) which use virGetGroupList and virFork. Reported by Coverity. 2017-10-12 Guido Günther virt-host-validate: require fuse for LXC if compiled in Domains fail to start without fuse like error: internal error: guest failed to start: fuse: device not found, try 'modprobe fuse' first Failure in libvirt_lxc startup: no error so check for it too. References: https://ci.debian.net/data/autopkgtest/unstable/amd64/libv/libvirt/20171012_105903/log.gz Reviewed-by: Daniel P. Berrange 2017-10-12 Chao Fan qemu: add the print of page size in cmd domjobinfo The command "info migrate" of qemu outputs the dirty-pages-rate during migration, but page size is different in different architectures. So page size should be output to calculate dirty pages in bytes. Page size is already implemented with commit 030ce1f8612215fcbe9d353dfeaeb2937f8e3f94 in qemu. Now Implement the counter-part in libvirt. 2017-10-12 Nitesh Konkar cpu_ppc64: Error out when model tag missing in virsh cpu-compare xml libvirtd throws unhandled signal 11 on ppc while running virsh cpu-compare with missing model tag in the xml. This patch errors out in such situation. 2017-10-12 caoxinhua Fix 1 << -1 in JOB_MASK macro Calling JOB_MASK(QEMU_JOB_NONE) would result in 1 << -1. 2017-10-11 Daniel P. Berrange maint: update to latest gnulib This pulls in the fix for getopt tests on Fedora >= 28 / glibc > 2.26.0 2017-10-11 Jiri Denemark spec: Install README.md Installing dead README symlink only is pretty useless. Reviewed-by: Andrea Bolognani 2017-10-11 Ján Tomko qemu: clarify error message for index 0 PIIX3 USB controller The address is restricted to 0:0:1.2 only for the piix3-uhci controller at index 0. https://bugzilla.redhat.com/show_bug.cgi?id=1460602 2017-10-11 Andrea Bolognani qemu: Don't crash when parsing command line lacking -M Parse the -M (or -machine) command line option before starting processing in earnest and have a fallback ready in case it's not present, so that while parsing other options we can rely on def->os.machine being initialized. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1379218 Reviewed-by: Daniel P. Berrange 2017-10-10 Marc Hartmayer lxc: Fixed indentation Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski lxc: Fixed a typo Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski 2017-10-10 Marc Hartmayer util: Fix deadlock across fork() This commit fixes the deadlock introduced by commit 0980764dee687e8da86dc410c351759867163389. The call getgrouplist() of the glibc library isn't safe to be called in between fork and exec (see commit 75c125641ac73473ba4b0542524d67a184769c8e). Fixes: 0980764dee68 ("util: share code between virExec and virCommandExec") Reviewed-by: Bjoern Walk Reviewed-by: Boris Fiuczynski 2017-10-10 Marc Hartmayer util: Add virCommandGetGID and virCommandGetUID These functions are used by an upcoming commit. Reviewed-by: Boris Fiuczynski 2017-10-10 Kothapally Madhu Pavan qemu: Remove redundant code in qemuParseCommandLineDisk 2017-10-07 Jim Fehlig apparmor: add dnsmasq ptrace rule to libvirtd profile Commit b482925c added ptrace rule for the apparmor profiles, but one was missed in the libvirtd profile for dnsmasq. It was overlooked since the test machine did not have an active libvirt network requiring dnsmasq that was also set to autostart. With one active and set to autostart, the following denial is observed in audit.log when restarting libvirtd type=AVC msg=audit(1507320136.306:298): apparmor="DENIED" \ operation="ptrace" profile="/usr/sbin/libvirtd" pid=5472 \ comm="libvirtd" requested_mask="trace" denied_mask="trace" \ peer="/usr/sbin/dnsmasq" With an active network, I suspect a libvirtd restart causes access to /proc//*, hence the resulting denial. As a nasty side affect of the denial, libvirtd thinks it needs to spawn a dnsmasq process even though one is already running for the network. E.g. after two libvirtd restarts dnsmasq 1683 0.0 0.0 51188 2612 ? S 12:03 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper root 1684 0.0 0.0 51160 576 ? S 12:03 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper dnsmasq 4706 0.0 0.0 51188 2572 ? S 13:54 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper root 4707 0.0 0.0 51160 572 ? S 13:54 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper dnsmasq 4791 0.0 0.0 51188 2580 ? S 13:56 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper root 4792 0.0 0.0 51160 572 ? S 13:56 0:00 \ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf \ --leasefile-ro --dhcp-script=/usr/lib64/libvirt/libvirt_leaseshelper A simple fix is to add a ptrace rule for dnsmasq. Reviewed-By: Guido Günther 2017-10-06 Wim ten Have numa: rename function virDomainNumaDefCPUFormat Rename virDomainNumaDefCPUFormat to virDomainNumaDefCPUFormatXML, matching its peer virDomainNumaDefCPUParseXML and the general vir*{Format,Parse}XML conventions. Reviewed-by: Jim Fehlig 2017-10-06 Wim ten Have build: isolate core libvirt libs deps from xen runtime Generating libvirt packages per make rpm, "with-libxl=1" and "with-xen=1", adds strict runtime dependencies per libxenlight for xen-libs package from core libvirt-libs package. This is not necessary and unfortunate since those dependencies set demand to "xen-libs" package even when there's no need for libvirt xen or libxl driver components. This patch is to have two separate xenconfig lib tool libraries: one for core libvirt (without XL), and a another that contains xl for libxl driver (libvirt_driver_libxl_impl.la) which when loading the driver, loads the remaining symbols (xen{Format,Parse}XL. For the user/sysadmin, this means the xen dependencies are moved into libxl driver, instead of core libvirt. Reviewed-by: Jim Fehlig 2017-10-06 John Ferlan storage: Use virStoragePoolObjGetDef accessor for Gluster backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for FS backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for Sheepdog backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for Logical backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for Disk backend In preparation for privatizing the object, use the accessor. storage: Use virStoragePoolObjGetDef accessor for storage_util In preparation for privatizing the object, use the accessor. test: Use virStoragePoolObjGetDef accessor In preparation for privatizing the object, use the accessor. test: Create local virStoragePoolObjPtr VolLookup APIs Rather than accessing privconn->pools.objs[i] in the for loop, let's use an @obj variable to make it easier to read the code. 2017-10-06 John Ferlan test: Rename @vol to @volDef in testOpenVolumesForPool Make it more obvious as we're about to need to change how obj->def gets referenced. Perform a couple of minor cleanups along the way too. 2017-10-06 John Ferlan storage: Use virStoragePoolObjGetDef accessor for driver In preparation for privatizing the object, use the accessor to fetch the obj->def instead of the direct reference. tests: Fix possible NULL deref conf: Fix prototype/definition for virStoragePoolObj get functions Modify virStoragePoolObjGetAutostartLink and virStoragePoolObjGetConfigFile to return "const char *" since that's how both are used and to ensure no one tries to VIR_FREE the result. 2017-10-06 Christian Ehrhardt virt-aa-helper: put static rules in quotes To avoid any issues later on if paths ever change (unlikely but possible) and to match the style of other generated rules the paths of the static rules have to be quoted as well. 2017-10-06 Christian Ehrhardt virt-aa-helper: allow spaces in vm names libvirt allows spaces in vm names, there were issues in the past but it seems not removed so the assumption has to be that spaces are continuing to be allowed. Therefore virt-aa-helper should not reject spaces in vm names anymore if it is going to be refused causing issues then the parser or xml schema should do so. Apparmor rules are in quotes, so a space in a path based on the name works. 2017-10-06 Christian Ehrhardt virt-aa-helper: fix libusb access to udev usb data libusb as used by qemu needs to read data from /run/udev/data/ about usb devices. That is read once on the first initialization of libusb_init by qemu. Therefore generating just the device we need would not be sufficient as another hotplug later can need another device which would fail as the data is no more re-read at this point. But we can restrict the paths very much to just the major number of potential usb devices which will make it match approximately the detail that e.g. an lsusb -v would reveal - that is much safer than the "/run/udev/data/* r" blanket many users are using now as a workaround. 2017-10-06 Christian Ehrhardt virt-aa-helper: fix paths for usb hostdevs If users only specified vendor&product (the common case) then parsing the xml via virDomainHostdevSubsysUSBDefParseXML would only set these. Bus and Device would much later be added when the devices are prepared to be added. Due to that a hot-add of a usb hostdev works as the device is prepared and virt-aa-helper processes the new internal xml. But on an initial guest start at the time virt-aa-helper renders the apparmor rules the bus/device id's are not set yet: p ctl->def->hostdevs[0]->source.subsys.u.usb $12 = {autoAddress = false, bus = 0, device = 0, vendor = 1921, product = 21888} That causes rules to be wrong: "/dev/bus/usb/000/000" rw, The fix calls virHostdevFindUSBDevice after reading the XML from virt-aa-helper to only add apparmor rules for devices that could be found and now are fully known to be able to write the rule correctly. It uncondtionally sets virHostdevFindUSBDevice mandatory attribute as adding an apparmor rule for a device not found makes no sense no matter what startup policy it has set. 2017-10-06 Peter Krempa qemu: process: Don't redetect backing chain on reconnect Skip purging the backing chain and redetecting it when it was not going to change during the time we were not present. The decision is based on the new flag which records whether there were blockjobs running to the status XML. 2017-10-06 Peter Krempa qemu: process: Simplify access to individual disk when reconnecting Add a helper variable so that we don't have to access the disk via 3 indirections. qemu: domain: Mark if no blockjobs are active in the status XML Note when no blockjobs are running in the status XML so that we know that the backing chain will not change until we reconnect. util: Add functions to simplify bool->virTristate(Bool|Switch) assignment virTristateBoolFromBool and virTristateSwitchFromBool convert a boolean to the correct enum value. 2017-10-06 Michal Privoznik news: Document watchdog hot-(un)plug change In 361c8dc17 and 662140fa68ae0 I've implemented hot-(un)plug of watchdog devices. Document this change. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik qemu: hot-unplug of watchdog https://bugzilla.redhat.com/show_bug.cgi?id=1447169 Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik qemu: hot-plug of watchdog https://bugzilla.redhat.com/show_bug.cgi?id=1447169 Since domain can have at most one watchdog it simplifies things a bit. However, since we must be able to set the watchdog action as well, new monitor command needs to be used. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik qemuDomainDeviceDefValidate: Validate watchdog Currently we don't do it. Therefore we accept senseless combinations of models and buses they are attached to. Moreover, diag288 watchdog is exclusive to s390(x). Reviewed-by: John Ferlan 2017-10-05 John Ferlan docs,rng: Adjust storage pool name grammar checks https://bugzilla.redhat.com/show_bug.cgi?id=1475250 It's possible to define and start a pool with a '.' in the name; however, when trying to add a volume to a domain using the storage pool source with a '.' in the storage pool name, the domain RNG validation fails because RNG uses 'genericName' which does not allow a '.' in the name. Domain XML def parsing has a virXMLValidateAgainstSchema which generates the error. The Storage Pool XML def parsing has no call to virXMLValidateAgainstSchema. The only Storage Pool name validation occurs in virStoragePoolDefParseXML to ensure the name doesn't have a '/' in it and in storagePoolDefineXML to call virXMLCheckIllegalChars using the same parameter "\n" as qemuDomainDefineXMLFlags would check after the RNG check could be succesful. In order to resolve this, create a poolName definition in storagecommon.rng that will mimic the domain name regex that disallows a newline character, but add the "/" in the exclude list. Then modify the pool and volume source name definitions to key off that poolName. 2017-10-05 Peter Krempa tests: qemuxml2xml: Simplify adding to the status XML expect string generator Unindent the static XML block and move around the autoindent calls so that further additions don't have to add more of them. Also rename the string holding the static XML section. 2017-10-05 Peter Krempa qemu: driver: Save status XML when starting a block job Few jobs (which don't) use the 'mirror' element did not save the status XML. It will be helpful to do so once we start tracking jobs fully. qemu: blockjob: Always save config XML when a blockjob is finished For VMs with persistent config the config may change upon successful completion of a job. Save it always if a persistent VM finishes a blockjob. This will simplify further additions. qemu: blockjob: Always save status XML after block event The status XML would be saved only for the copy job (in case of success) or on failure even for other jobs. As the status contains the backing chain data, which change after success we should always save it on block job completion. 2017-10-05 Michal Privoznik news: Document domifstat and QoS changes made recently In cea3715b2e9 and d86fd2402e9d2 I've fixed domifstat and QoS that was reversed for some types of interfaces. Document this in the news file. 2017-10-05 Jiri Denemark spec: Enable qemu driver on s390x on RHEL/CentOS 2017-10-05 Peter Krempa qemu: process: move disk presence checking to host setup function Checking of disk presence accesses storage on the host so it should be done from the host setup function. Move the code to new function called qemuProcessPrepareHostStorage and remove qemuDomainCheckDiskPresence. qemu: process: Move TLS setup for storage source to qemuProcessPrepareDomainStorage 2017-10-05 Peter Krempa qemu: process: Move 'volume' translation to domain prepare stage Introduce a new function to prepare domain disks which will also do the volume source to actual disk source translation. The 'pretend' condition is not transferred to the new location since it does not help in writing tests and also no tests abuse it. 2017-10-05 Peter Krempa qemu: domain: Document and export qemuDomainCheckDiskStartupPolicy qemu: process: Pass flags to qemuProcessPrepareHost Pass flags to the function rather than just whether we have incoming migration. This also enforces correct startup policy for USB devices when reverting from a snapshot. qemu: migration: Extract flags for starting VM into a variable qemuMigrationPrepareAny called multiple of the functions starting the qemu process for incoming migration by adding the flags explicitly. Extract them to a variable so that they can be easily used for other calls or changed in the future. qemu: process: document parameters for startup preparing functions Document mainly what flag values are passed in. qemu: command: Move PPC fdc check to domain validation Apart from not littering the command line generator, the added benefit is that new configs with a FDC will be rejected at define stage. qemu: command: Separate validation from command line building for -drive Remove validation code into a separate function so that it's not interleaved with actual building of the command line. 2017-10-05 Michal Privoznik libvirt-domain: Document interface stats POV Interestingly enough, we don't document the point of view of the interface statistics. Therefore it's unknown to users if for instance rx_packets is the number of packets received by domain or received by host (from domain). Document this explicitly. 2017-10-05 Michal Privoznik QoS: Set classes and filters in proper direction Similarly to previous patch, for some types of interface domain and host are on the same side of RX/TX barrier. In that case, we need to set up the QoS differently. Well, swapped. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik virNetDevTapInterfaceStats: Allow caller to not swap the statistics https://bugzilla.redhat.com/show_bug.cgi?id=1497410 The comment in virNetDevTapInterfaceStats() implementation for Linux states that packets transmitted by domain are received by the host and vice versa. Well, this is true but not for all types of interfaces. For instance, for macvtaps when TAP device is hooked right onto a physical device any packet that domain sends looks also like a packet sent to the host. Therefore, we should allow caller to chose if the stats returned should be straight copy or swapped. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik src: Use virDomainNetFindByName Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik conf: Introduce virDomainNetFindByName Small wrapper to lookup interface in domain definition by its name. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik qemuDomainInterfaceStats: Check for the actual type of interface Users might have configured interface so that it's type of network, but the corresponding network plugs interfaces into an OVS bridge. Therefore, we have to check for the actual type of the interface instead of the configured one. Reviewed-by: John Ferlan 2017-10-05 Michal Privoznik lxc: Drop useless ifdef __linux__ This code compiles only on Linux. Therefore the condition we check is always true. Reviewed-by: John Ferlan 2017-10-05 Jiri Denemark tests: Fix build with clang clang doesn't like mode_t type as an argument to va_arg(): error: second argument to 'va_arg' is of promotable type 'mode_t' (aka 'unsigned short'); this va_arg has undefined behavior because arguments will be promoted to 'int' mode = va_arg(ap, mode_t); ^~~~~~ 2017-10-04 Pavel Hrdina storage: Fix incorrect parenthesis placement Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1498528 2017-10-04 Lin Ma qemu: Support multiqueue virtio-blk qemu 2.7.0 introduces multiqueue virtio-blk(commit 2f27059). This patch introduces a new attribute "queues". An example of the XML: The corresponding QEMU command line: -device virtio-blk-pci,scsi=off,num-queues=4,id=virtio-disk0 2017-10-04 Lin Ma qemucapstest: Update test data for 'num-queues' property of virtio-blk 2017-10-04 Michal Privoznik virDomainNetFindIdx: Ignore auto generated MAC addresses When detaching an from a domain, the MAC address is parsed and if not present one is generated. If no corresponding interface is found in the domain, the following error is reported: error: operation failed: no device matching mac address 52:54:00:75:32:5b found where the MAC address is the auto generated one. This might be very confusing. Solution to this is to ignore auto generated MAC address when looking up the device. Reviewed-by: Erik Skultety 2017-10-04 Michal Privoznik virmacaddr: Track if MAC address is autogenerated It will come handy to know if the MAC address was generated (e.g. during XML parse) or if it was parsed since provided by user in the XML. Reviewed-by: Erik Skultety 2017-10-04 Michal Privoznik qemuParseCommandLineNet: Make it more readable Reviewed-by: Erik Skultety 2017-10-04 John Ferlan nwfilter: Fix memory leak and error path Found by Coverity. If virNWFilterHashTablePut, then the 3rd arg @val must be free'd since it would be leaked. This also fixes potential problem on the error path where the caller could assume the virNWFilterHashTablePut was successful when in fact it failed leading to other issues. 2017-10-04 John Ferlan nwfilter: Clean up virNWFilterDetermineMissingVarsRec returns Rather than using loop break;'s in order to force a return of rc = -1, let's just return -1 immediately on the various error paths and then return 0 on the success path. 2017-10-04 Luyao Huang tests: Do not ignore mode parameter in mocked open() This is normally not an issue since the tests which use mocked open() do not create files. But once coverage build is enabled, gcov_open will use O_CREATE and real_open will read random data rather than the actual mode argument. 2017-10-04 Kothapally Madhu Pavan util: Free a pointer in virPolkitCheckAuth Free DBusMessage pointer in virPolkitCheckAuth 2017-10-04 Peter Krempa conf: Split out parsing of network disk source XML elements virDomainDiskSourceParse got to the point of being an ugly spaghetti mess by adding more and more stuff into it. Split out parsing of network disk information into a separate function so that it stays contained. Post-release version bump to 3.9.0 2017-10-04 Daniel Veillard Release of libvirt-3.8.0 * docs/news.xml: updated for release * po/*.po*: regenerated 2017-10-03 Jiri Denemark spec: Delay supported_platform check Building RPM should only be allowed on a supported platform, but unpacking the source and applying all patches can be done anywhere. 2017-10-03 Martin Kletzander docs: Document the real behaviour of suspend-to-{mem,disk} We get a question every now and then about why hibernation works when suspend-to-disk is disabled and similar. Let's hope that, by documenting the obvious more blatantly, people will get more informed. Reviewed-by: Erik Skultety 2017-10-02 John Ferlan nwfilter: Don't have virNWFilterIPAddrMapAddIPAddr consume input On pure success paths, virNWFilterIPAddrMapAddIPAddr was validly consuming the input @addr; however, on failure paths it was possible that virNWFilterVarValueCreateSimple succeed, but virNWFilterHashTablePut failed resulting in virNWFilterVarValueFree being called to clean up @val which also cleaned up the input @addr. Thus the caller had no way to determine on failure whether it too should clean up the passed parameter. Instead, let's create a copy of the input @addr, then handle that properly in the API allowing/forcing the caller to free it's own copy of the input parameter. 2017-10-02 John Ferlan Revert "nwfilter: Fix possible segfault on sometimes consumed variable" This reverts commit 6209bb32e5b6d8c15d55422bb4716b3b31c1c7b2. This turns out to be the wrong adjustment 2017-10-02 Martin Kletzander docs: Add some changes to news.xml for this release Reviewed-by: Andrea Bolognani 2017-09-28 Daniel P. Berrange Fix vxhs test to have stable certificate dir The test suite has hardcoded /etc/pki/qemu as the cert dir, but this only works if configure has --sysconfdir=/etc passed. We must set the vxhs cert dir to a stable path in the test suite. 2017-09-28 Ashish Mittal qemu: Add TLS support for Veritas HyperScale (VxHS) Alter qemu command line generation in order to possibly add TLS for a suitably configured domain. Sample TLS args generated by libvirt - -object tls-creds-x509,id=objvirtio-disk0_tls0,dir=/etc/pki/qemu,\ endpoint=client,verify-peer=yes \ -drive file.driver=vxhs,file.tls-creds=objvirtio-disk0_tls0,\ file.vdisk-id=eb90327c-8302-4725-9e1b-4e85ed4dc251,\ file.server.type=tcp,file.server.host=192.168.0.1,\ file.server.port=9999,format=raw,if=none,\ id=drive-virtio-disk0,cache=none \ -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,\ id=virtio-disk0 Update the qemuxml2argvtest with a couple of examples. One for a simple case and the other a bit more complex where multiple VxHS disks are added where at least one uses a VxHS that doesn't require TLS credentials and thus sets the domain disk source attribute "tls = 'no'". Update the hotplug to be able to handle processing the tlsAlias whether it's to add the TLS object when hotplugging a disk or to remove the TLS object when hot unplugging a disk. The hot plug/unplug code is largely generic, but the addition code does make the VXHS specific checks only because it needs to grab the correct config directory and generate the object as the command line would do. 2017-09-28 John Ferlan qemu: Introduce qemuDomainPrepareDiskSource Introduce a function to setup any TLS needs for a disk source. If there's a configuration or other error setting up the disk source for TLS, then cause the domain startup to fail. For VxHS, follow the chardevTLS model where if the src->haveTLS hasn't been configured, then take the system/global cfg->haveTLS setting for the storage source *and* mark that we've done so via the tlsFromConfig setting in storage source. Next, if we are using TLS, then generate an alias into a virStorageSource 'tlsAlias' field that will be used to create the TLS object and added to the disk object in order to link the two together for QEMU. 2017-09-28 Ashish Mittal util: Add TLS attributes to virStorageSource Add an optional virTristateBool haveTLS to virStorageSource to manage whether a storage source will be using TLS. Sample XML for a VxHS disk: Additionally add a tlsFromConfig boolean to control whether the TLS setting was due to domain configuration or qemu.conf global setting in order to decide whether to Format the haveTLS setting for either a live or saved domain configuration file. Update the qemuxml2xmltest in order to add a test to show the proper parsing. Also update the docs to describe the tls attribute. 2017-09-28 John Ferlan docs: Clean up the description for network disk protocol options Clean up the description a bit to make it more readable and not appear as one long run-on paragraph. 2017-09-28 Ashish Mittal conf: Introduce TLS options for VxHS block device clients Add a new TLS X.509 certificate type - "vxhs". This will handle the creation of a TLS certificate capability for properly configured VxHS network block device clients. The following describes the behavior of TLS for VxHS block device: (1) Two new options have been added in /etc/libvirt/qemu.conf to control TLS behavior with VxHS block devices "vxhs_tls" and "vxhs_tls_x509_cert_dir". (2) Setting "vxhs_tls=1" in /etc/libvirt/qemu.conf will enable TLS for VxHS block devices. (3) "vxhs_tls_x509_cert_dir" can be set to the full path where the TLS CA certificate and the client certificate and keys are saved. If this value is missing, the "default_tls_x509_cert_dir" will be used instead. If the environment is not configured properly the authentication to the VxHS server will fail. 2017-09-28 John Ferlan nwfilter: Fix possible segfault on sometimes consumed variable The virNWFilterIPAddrMapAddIPAddr code can consume the @addr parameter on success when the @ifname is found in the ipAddressMap->hashTable hash table in the call to virNWFilterVarValueAddValue; however, if not found in the hash table, then @addr is formatted into a @val which is stored in the table and on return the caller would be expected to free @addr. Thus, the caller has no way to determine on success whether @addr was consumed, so in order to fix this create a @tmp variable which will be stored/consumed when virNWFilterVarValueAddValue succeeds. That way the caller can free @addr whether the function returns success or failure. 2017-09-27 Pavel Hrdina rpc: for messages with FDs always decode count of FDs from the message The packet with passed FD has the following format: -------------------------- | len | header | payload | -------------------------- where "payload" has an additional count of FDs before the actual data: ------------------ | nfds | payload | ------------------ When the packet is received we parse the "header", which as a side effect updates msg->bufferOffset to point to the beginning of "payload". If the message call contains FDs, we need to also parse the count of FDs, which also updates the msg->bufferOffset. The issue here is that when we attempt to read the FDs data from the socket and we receive EAGAIN we finish the reading and call poll() to wait for the data the we need. When the data arrives we already have the packet in our buffer so we read the "header" again but this time we don't read the count of FDs because we already have it stored. That means that the msg->bufferOffset is not updated to point to the actual beginning of the payload data, but it points to the count of FDs. After all FDs are processed we dispatch the message to process it and decode the payload. Since the msg->bufferOffset points to wrong data, we decode the wrong payload and the API call fails with error messages: Domain not found: no domain with matching uuid '67656e65-7269-6300-0c87-5003ca6941f2' () Broken by commit 133c511b527 which fixed a FD and memory leak. 2017-09-27 Peter Krempa qemu: domain: Extract common clearing of VM private data VM private data is cleared when the VM is turned off and also when the VM object is being freed. Some of the clearing code was duplicated. Extract it to a separate function. This also removes the now unnecessary function qemuDomainClearPrivatePaths. 2017-09-27 Erik Skultety docs: Add the closing
tag for mdev example 2017-09-27 Ján Tomko Shrink volume even with ALLOCATE flag Calling fallocate on the new (smaller) capacity ensures that the whole file is allocated, but it does not reduce the file size. Also call ftruncate after fallocate. https://bugzilla.redhat.com/show_bug.cgi?id=1366446 2017-09-27 Ján Tomko virStorageFileResize: fallocate the whole capacity We have been trying to implement the ALLOCATE flag to mean "the volume should be fully allocated after the resize". Since commit b0579ed9 we do not allocate from the existing capacity, but from the existing allocation value. However this value is a total of all the allocated bytes, not an offset. For a sparsely allocated file: $ perl -e 'print "x"x8192;' > vol1 $ fallocate -p -o 0 -l 4096 vol1 $ virsh vol-info vol1 default Capacity: 8.00 KiB Allocation: 4.00 KiB Treating allocation as an offset would result in an incompletely allocated file: $ virsh vol-resize vol1 --pool default 16384 --allocate Capacity: 16.00 KiB Allocation: 12.00 KiB Call fallocate from zero on the whole requested capacity to fully allocate the file. After that, the volume is fully allocated after the resize: $ virsh vol-resize vol1 --pool default 16384 --allocate $ virsh vol-info vol1 default Capacity: 16.00 KiB Allocation: 16.00 KiB 2017-09-27 Ján Tomko use virFileAllocate in virStorageFileResize Introduce a new function virFileAllocate that will call the non-destructive variants of safezero, essentially reverting my commit 1390c268 safezero: fall back to writing zeroes even when resizing back to the state as of commit 18f0316 virstoragefile: Have virStorageFileResize use safezero This means that _ALLOCATE flag will no longer work on platforms without the allocate syscalls, but it will not overwrite data either. 2017-09-27 John Ferlan virsh: Add/allow secret-uuid for pool-{define|create}-as https://bugzilla.redhat.com/show_bug.cgi?id=1476775 For the virsh pool-{define|create}-as command, let's allow using --secret-uuid on the command line as an alternative to --secret-usage (added for commit id '8932580'), but ensure that they are mutually exclusive. 2017-09-27 ZhiPeng Lu nwfilter: Fix memory leak in learnIPAddressThread Don't leak @inetaddr within the done: processing when attempting to instantiate the filter. nwfilter: Fix memory leak in virNWFilterIPAddrMapAddIPAddr If virNWFilterHashTablePut fails, then the @val was leaked. 2017-09-27 Ján Tomko qemu: fix hotplug of udp device with no connect host Use an empty string to let qemu fill out the default. This matches what's done in qemuBuildChrChardevStr. https://bugzilla.redhat.com/show_bug.cgi?id=1454671 Reviewed-by: John Ferlan 2017-09-27 Pavel Hrdina Revert "vhost-user: add support reconnect for vhost-user ports" This reverts commit edaf4ebe95a5995585c8ab7bc5b92887286d4431. This uses "reconnect" as attribute for element, but we already have a element for element for chardev devices. Since this is the same feature for different device it should be presented in XML the same way. 2017-09-27 Peter Krempa qemu: process: Refresh data from qemu monitor after migration Some values we read from the qemu monitor may be changed with the actual state by the incoming migration. This means that we should refresh certain things only after the migration has finished. This is mostly visible in the cdrom tray state, which is by default closed but may be opened by the guest OS. This would be refreshed before qemu transferred the actual state and thus libvirt would think that the tray is closed. Note that this patch moves only a few obvious query commands. Others may be moved later after individual assessment. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1463168 2017-09-27 Peter Krempa qemu: hotplug: Ignore cgroup errors when hot-unplugging vcpus When the vcpu is successfully removed libvirt would remove the cgroup. In cases when removal of the cgroup fails libvirt would report an error. This does not make much sense, since the vcpu was removed and we can't really do anything with the cgroup. This patch silences the errors from cgroup removal. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1462092 2017-09-26 Ján Tomko qemu: adjust indentation of qemuDomainObjPrivateXMLFormatAutomaticPlacement Commit 6801da94 fixed the typo in the function name, but forgot to adjust the indentation level of the next line. 2017-09-26 Peter Krempa qemu: domain: Fix typo in qemuDomainObjPtrivateXMLFormatAutomaticPlacement 2017-09-26 Ján Tomko conf: fix formatting of udp chardev attributes It is possible (although possibly not very useful) to leave out the service attribute when using Fix the formatter bug introduced by commit 4a0da34 and format the host when its present (checked for non-NULL inside virBufferEscapeString) instead of basing it on the presence of the service attribute. https://bugzilla.redhat.com/show_bug.cgi?id=1455825 2017-09-25 Peter Krempa qemu: block: Use correct alias when extracting disk node names The alias recorded in disk->info.alias is the alias for the frontend device but we are interested in the backend drive. This messed up the disk node name extraction code as qemu reports the drive alias in the block query commands. This was broken in the node name detector refactoring done in commit 0175dc6ea024d Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1494327 2017-09-25 Peter Krempa qemu: block: Don't lookup node names if they are already known Move the check that skips node name detection if they are already present earlier so that the hash table lookup is skipped. 2017-09-25 Daniel P. Berrange Print hex values with '0x' prefix and octal with '0' in debug messages Seeing a log message saying 'flags=93' is ambiguous & confusing unless you happen to know that libvirt always prints flags as hex. Change our debug messages so that they always add a '0x' prefix when printing flags, and '0' prefix when printing mode. A few other misc places gain a '0x' prefix in error messages too. 2017-09-25 Jim Fehlig apparmor: support ptrace checks Kernel 4.13 introduced finer-grained ptrace checks https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.13.2&id=290f458a4f16f9cf6cb6562b249e69fe1c3c3a07 With kernel 4.13 and apparmor 2.11, simply starting libvirtd results in the following apparmor denial type=AVC msg=audit(1506112085.645:954): apparmor="DENIED" operation="ptrace" profile="/usr/sbin/libvirtd" pid=6984 comm="libvirtd" requested_mask="trace" denied_mask="trace" peer="unconfined" Attempting to start an unconfined domain results in type=AVC msg=audit(1506112301.227:1112): apparmor="DENIED" operation="ptrace" profile="/usr/sbin/libvirtd" pid=7498 comm="libvirtd" requested_mask="trace" denied_mask="trace" peer="/usr/sbin/libvirtd" And attempting to start a confined domain results in type=AVC msg=audit(1506112631.408:1312): apparmor="DENIED" operation="open" profile="virt-aa-helper" name="/etc/libnl/classid" pid=8283 comm="virt-aa-helper" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 type=AVC msg=audit(1506112631.530:1319): apparmor="DENIED" operation="open" profile="virt-aa-helper" name="/etc/libnl/classid" pid=8289 comm="virt-aa-helper" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 type=AVC msg=audit(1506112632.186:1324): apparmor="DENIED" operation="ptrace" profile="/usr/sbin/libvirtd" pid=8342 comm="libvirtd" requested_mask="trace" denied_mask="trace" peer="libvirt-66154842-e926-4f92-92f0-1c1bf61dd1ff" Add ptrace rules to allow the trace operations. Resolves: https://bugzilla.suse.com/show_bug.cgi?id=1058847 Reviewed-by: Guido Günther 2017-09-22 Ján Tomko news: remove kernel version reference from switchdev entry The functionality was added in 4.8, but due to a rename of the DEVLINK_CMD_ESWITCH_GET constant in the kernel headers, the headers from kernel 4.11 are required by the libvirt code. Remove the reference from the news entry, since it could be misleading. 2017-09-22 Peter Krempa qemu: capabilities: Remove support for downstream-only QMP monitor backport Some distros (see diff) chose to backport QMP support rather than rebase to newer version of qemu. As a hack they added the string 'libvirt' to the qemu -help output. Remove this as downstream-only hacks should be carried by downstream and not litter upstream. This effectively reverts commit ff88cd590572277f10ecee4ebb1174d9b70fc0d7 2017-09-22 Jiri Denemark qemu: Use qemuDomainDefFormatXML in qemuDomainDefCopy Because qemuDomainDefCopy needs a string representation of a domain definition, there's no reason for calling the lower level qemuDomainDefFormatBuf API. qemu: Fix error checking in qemuDomainDefFormatXMLInternal virDomainDefFormatInternal (called by qemuDomainDefFormatXMLInternal) already checks for buffer errors and properly resets the buffer on failure. 2017-09-22 Michal Privoznik libvirt_nss.c: Fix typo in aiforaf() In my previous commit of b1d87f9ad96f I've made a typo breaking the FreeBSD build. s/ipAaddr/ipAddr/ 2017-09-22 Guido Günther apparmor: delete profile on VM shutdown instead of only unloading it. This makes sure old profiles don't pile up in /etc/apparmor.d/libvirt and we get updates to modified templates on VM restart. Reviewed-by: Jim Fehlig 2017-09-22 Laine Stump util: Fix stack smashing in virNetDevGetFamilyId After commit 8708ca01c0d libvirtd consistently aborts with "stack smashing detected" when nodedev driver is initialized. This is caused by nlmsg_parse() being told that its array of nlattr* has CTRL_CMD_MAX (10) entries, when in fact it is declared to have CTRL_ATTR_MAX (8) entries. Since all the entries are initialized to NULL, the result is that nlmsg_parse is overwriting 2*(sizof(nlattr*)) bytes outside the array. Reviewed-by: John Ferlan 2017-09-22 Michal Privoznik nss: Don't leak @ipAddr In aiforaf() (which exists only when building for BSD) the @ipAddr may be leaked. 2017-09-21 John Ferlan util: Fix secret generation in virStorageSourceParseRBDColonString Commit id '5604c056' used the wrong API to generate the qemu: Avoid a possible NULL pointer deref in qemuDomainGetTLSObjects Passing a NULL value for the argument secAlias to the function qemuDomainGetTLSObjects would cause a segmentation fault in libvirtd. Changed code to check before dereferencing a NULL secAlias. 2017-09-21 Boris Fiuczynski tests: Add QEMU 2.10.0 capabilities test for s390x Adding s390x qemu caps test for qemu version 2.10.0. 2017-09-21 Michal Privoznik qemu: Introduce a wrapper over virFileWrapperFdClose https://bugzilla.redhat.com/show_bug.cgi?id=1448268 When migrating to a file (e.g. when doing 'virsh save file'), couple of things are happening in the thread that is executing the API: 1) the domain obj is locked 2) iohelper is spawned as a separate process to handle all I/O 3) the thread waits for iohelper to finish 4) the domain obj is unlocked Now, the problem is that while the thread waits in step 3 for iohelper to finish this may take ages because iohelper calls fdatasync(). And unfortunately, we are waiting the whole time with the domain locked. So if another thread wants to jump in and say copy the domain name ('virsh list' for instance), they are stuck. The solution is to unlock the domain whenever waiting for I/O and lock it back again when it finished. Reviewed-by: John Ferlan 2017-09-21 John Ferlan qemu: Be more selective when determining cdrom for taint messaging https://bugzilla.redhat.com/show_bug.cgi?id=1471225 Commit id '99a2d6af2' was a bit too aggressive with determining whether the provided path was a "physical" cd-rom in order to generate a taint message due to the possibility of some guest and host trying to control the tray. For cd-rom guest devices backed to some VIR_STORAGE_TYPE_FILE storage, this wouldn't be a problem and as such it shouldn't be a problem for guest devices using some sort of block device on the host such as iSCSI, LVM, or a Disk pool would present. So before issuing a taint message, let's check if the provided path of the VIR_STORAGE_TYPE_BLOCK backed device is a "known" physical cdrom name by comparing the beginning of the path w/ "/dev/cdrom" and "/dev/sr". Also since it's possible the provided path could resolve to some /dev/srN device, let's get that path as well and perform the same check. 2017-09-21 Michal Privoznik qemuBuildHostNetStr: Don't leak @addr The virSocketAddrFormat() allocates the string and it's caller responsibility to free it afterwards. ==28857== 11 bytes in 1 blocks are definitely lost in loss record 37 of 168 ==28857== at 0x4C2BEDF: malloc (vg_replace_malloc.c:299) ==28857== by 0x9A81D79: strdup (in /lib64/libc-2.23.so) ==28857== by 0x5DA3BF0: virStrdup (virstring.c:902) ==28857== by 0x5D96182: virSocketAddrFormatFull (virsocketaddr.c:427) ==28857== by 0x5D95E13: virSocketAddrFormat (virsocketaddr.c:352) ==28857== by 0x5706890: qemuBuildHostNetStr (qemu_command.c:3891) ==28857== by 0x57138D3: qemuBuildInterfaceCommandLine (qemu_command.c:8597) ==28857== by 0x5713D6A: qemuBuildNetCommandLine (qemu_command.c:8699) ==28857== by 0x57176F6: qemuBuildCommandLine (qemu_command.c:10027) ==28857== by 0x5769D61: qemuProcessCreatePretendCmd (qemu_process.c:6004) ==28857== by 0x4056EC: testCompareXMLToArgv (qemuxml2argvtest.c:502) ==28857== by 0x41DF40: virTestRun (testutils.c:180) Reviewed-by: John Ferlan 2017-09-21 Jiri Denemark qemu: Don't update CPU when formatting live def Since commit v2.2.0-199-g7ce711a30e libvirt stores an updated guest CPU in domain's live definition and there's no need to update it every time we want to format the definition. The commit itself tried to address this in qemuDomainFormatXML, but forgot to fix qemuDomainDefFormatLive. Not to mention that masking a previously set flag is only acceptable if the flag was set by a public API user. Internally, libvirt should have never set the flag in the first place. https://bugzilla.redhat.com/show_bug.cgi?id=1485022 2017-09-21 Jiri Denemark qemu: Use correct host model for updating guest cpu When a user requested a domain XML description with VIR_DOMAIN_XML_UPDATE_CPU flag, libvirt would use the host CPU definition from host capabilities rather than the one which will actually be used once the domain is started. https://bugzilla.redhat.com/show_bug.cgi?id=1481309 2017-09-21 Jiri Denemark conf: Drop unused VIR_DOMAIN_DEF_FORMAT_UPDATE_CPU The only real usage of this flag was removed by "cpu_conf: Drop updateCPU from virCPUDefFormat". cpu_conf: Simplify formatting of guest CPU attributes cpu_conf: Drop updateCPU from virCPUDefFormat In the past we updated host-model CPUs with host CPU data by adding a model and features, but keeping the host-model mode. And since the CPU model is not normally formatted for host-model CPU defs, we had to pass the updateCPU flag to the formatting code to be able to properly output updated host-model CPUs. Libvirt doesn't do this anymore, host-model CPUs are turned into custom mode CPUs once updated with host CPU data and thus there's no reason for keeping the hacks inside CPU XML formatters. qemuxml2xmltest: Add tests for Power CPUs 2017-09-21 Pino Toscano qemu: reject parallel ports for pseries machines They are simply not supported on that machine type. Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1487499 2017-09-21 Pino Toscano qemu: reject parallel ports for s390 archs They are simply not supported on those architectures. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1487499 2017-09-21 Pino Toscano qemu: pass the virDomainDef to qemuDomainChrDefValidate This will be used to improve the validation for this type of devices. The former @def parameter is renamed to @dev, leaving @def for the virDomainDef (following the style used elsewhere). 2017-09-21 Pino Toscano tests: qemuxml2argv: fail also on unexpected pass If a test expects either a parse error or a failure but then there is neither a parse error nor a failure, then properly mark the test as failing, instead of failing later on (e.g. trying to open a non-existing .args file). 2017-09-21 Pino Toscano tests: qemuxml2argv: fix expected type for usb-bus-missing The guest of usb-bus-missing does not cause a parse error, but a validation issue -- hence, switch from DO_TEST_PARSE_ERROR to DO_TEST_FAILURE. Fixes commit b003b9781b6ae633cfe4fdf6b9620ca246fa2432. 2017-09-21 Daniel P. Berrange Fix commandhelper build on win32 For win32 we need EXIT_AM_SKIP which is in testutils.h. We must define NO_LIBVIRT to prevent replacement of fprintf with virFilePrintf as we can't link to libvirt_util.la 2017-09-21 Daniel P. Berrange iohelper: avoid calling read() with misaligned buffers for O_DIRECT The iohelper currently calls saferead() to get data from the underlying file. This has a problem with O_DIRECT when hitting end-of-file. saferead() is asked to read 1MB, but the first read() it does may return only a few KB, so it'll try another read() to fill the remaining buffer. Unfortunately the buffer pointer passed into this 2nd read() is likely not aligned to the extent that O_DIRECT requires, so rather than seeing '0' for end-of-file, we'll get -1 + EINVAL due to misaligned buffer. The way the iohelper is currently written, it already handles getting short reads, so there is actually no need to use saferead() at all. We can simply call read() directly. The benefit of this is that we can now write() the data immediately so when we go into the subsequent reads() we'll always have a correctly aligned buffer. Technically the file position ought to be aligned for O_DIRECT too, but this does not appear to matter when at end-of-file. Tested-by: Nikolay Shirokovskiy Reviewed-by: Eric Blake 2017-09-20 Daniel P. Berrange Stop linking tests/commandhelper to libvirt code The commandhelper binary is a helper for commandtest that validates what file handles were inherited. For this to work reliably we must not have any libraries that leak file descriptors into commandhelper. Unfortunately some versions of gnutls will intentionally open file handles at library load time via a constructor function. We previously hacked around this in commit 4cbc15d037e1cd8abf5c4aa6acc30d83ae13e34d Author: Martin Kletzander Date: Fri May 2 09:55:52 2014 +0200 tests: don't fail with newer gnutls gnutls-3.3.0 and newer leaves 2 FDs open in order to be backwards compatible when it comes to chrooted binaries [1]. Linking commandhelper with gnutls then leaves these two FDs open and commandtest fails thanks to that. This patch does not link commandhelper with libvirt.la, but rather only the utilities making the test pass. Based on suggestion from Daniel [2]. [1] http://lists.gnutls.org/pipermail/gnutls-help/2014-April/003429.html [2] https://www.redhat.com/archives/libvir-list/2014-April/msg01119.html That fix relied on fact that while libvirt.so linked with gnutls, libvirt_util.la did not link to it. With the introduction of the util/vircrypto.c file that assumption is no longer valid. We must not link to libvirt_util.la at all - only gnulib and libc can (hopefully) be relied on not to open random file descriptors in constructors. Reviewed-by: Martin Kletzander 2017-09-20 ZhiPeng Lu vhost-user: add support reconnect for vhost-user ports For vhost-user ports, Open vSwitch acts as the server and QEMU the client. When OVS crashed or restart, QEMU shoule be reconnect to OVS. 2017-09-20 Julio Faracco storage: Add new events for *PoolBuild() and *PoolDelete(). This commit adds new events for two methods and operations: *PoolBuild() and *PoolDelete(). Using the event-test and the commands set below we have the following outputs: $ sudo ./event-test Registering event callbacks myStoragePoolEventCallback EVENT: Storage pool test Defined 0 myStoragePoolEventCallback EVENT: Storage pool test Created 0 myStoragePoolEventCallback EVENT: Storage pool test Started 0 myStoragePoolEventCallback EVENT: Storage pool test Stopped 0 myStoragePoolEventCallback EVENT: Storage pool test Deleted 0 myStoragePoolEventCallback EVENT: Storage pool test Undefined 0 Another terminal: $ sudo virsh pool-define test.xml Pool test defined from test.xml $ sudo virsh pool-build test Pool test built $ sudo virsh pool-start test Pool test started $ sudo virsh pool-destroy test Pool test destroyed $ sudo virsh pool-delete test Pool test deleted $ sudo virsh pool-undefine test Pool test has been undefined This commits can be a solution for RHBZ #1475227. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1475227 2017-09-20 ZhiPeng Lu qemu: handle reconnect on chardev hotplug The patch passes the reconnect timeout to QEMU by monitor on chardev hotplug. 2017-09-20 Daniel P. Berrange Link libvirt_util.la with gnutls The util/vircrypto.c file uses gnutls, so we must directly link libvirt_util.la with gnutls to avoid errors on OS which do not resolve symbols against indirectly linked libraries. This fixes a build failure on Ubuntu Trusty CCLD storagevolxml2argvtest /usr/bin/ld: ../src/.libs/libvirt_util.a(libvirt_util_la-vircrypto.o): undefined reference to symbol 'gnutls_strerror@@GNUTLS_1_4' //usr/lib/x86_64-linux-gnu/libgnutls.so.26: error adding symbols: DSO missing from command line 2017-09-20 Ashish Mittal qemu: Add qemu command line generation for a VxHS block device The VxHS block device will only use the newer formatting options and avoid the legacy URI syntax. An excerpt for a sample QEMU command line is: -drive file.driver=vxhs,file.vdisk-id=eb90327c-8302-4725-9e1b-4e85ed4dc251,\ file.server.type=tcp,file.server.host=192.168.0.1,\ file.server.port=9999,format=raw,if=none,id=drive-virtio-disk0,cache=none \ -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,\ id=virtio-disk0 Update qemuxml2argvtest with a simple test. 2017-09-20 Ashish Mittal qemu: Refactor qemuBlockStorageSourceBuildHostsJSONSocketAddress Extract out the "guts" of building a server entry into it's own separately callable/usable function in order to allow building a server entry for a consumer with src->nhosts == 1. util: storage: Add JSON backing volume parse for VxHS Add the backing parse and a test case to verify parsing of VxHS backing storage. 2017-09-20 Ashish Mittal docs: Add schema and docs for Veritas HyperScale (VxHS) Alter the schema to allow a VxHS block device. Sample XML is: eb90327c-8302-4725-9e1b-4e85ed4dc251
Update the html docs to describe the capability for VxHS. Alter the qemuxml2xmltest to validate the formatting. 2017-09-20 Ashish Mittal storage: Introduce VIR_STORAGE_NET_PROTOCOL_VXHS Add a new virStorageNetProtocol for Veritas HyperScale (VxHS) disks 2017-09-20 John Ferlan qemu: Detect support for vxhs Using the query-qmp-schema introspection - look for the 'vxhs' blockdevOptions type. NB: This is a "best effort" type situation as there is not a mechanism to determine whether the running QEMU has been built with '--enable-vxhs'. All we can do is check if the option to use vxhs for a blockdev-add exists in the command infrastructure which does not take that into account when building its table of commands and options. 2017-09-20 Laine Stump util: virPCIGetNetName(): use first netdev name when phys_port_id isn't matched The mlx4 (Mellanox) netdev driver implements the sysfs phys_port_id file for both VFs and PFs, so you can find the VF netdev plugged into the same physical port as any given PF netdev by comparing the contents of phys_port_id of the respective netdevs. That's what libvirt does when attempting to find the PF netdev for a given VF netdev (or vice versa). Most other netdev's drivers don't implement phys_port_id, so the file is visible in sysfs directory listing, but attempts to read it result in ENOTSUPP. In these cases, libvirt is unable to read phys_port_id of either the PF or the VF, so it just returns the first entry in the PF/VF's list of netdevs. But we've found that the i40e driver is in between those two situations - it implements phys_port_id for PF netdevs, but doesn't implement it for VF netdevs. So libvirt would successfully read the phys_port_id of the PF netdev, then try to find a VF netdev with matching phys_port_id, but would fail because phys_port_id is NULL for all VFs. This would result in a message like the following: Could not find network device with phys_port_id '3cfdfe9edc39' under PCI device at /sys/class/net/ens4f1/device/virtfn0 To solve this problem in a way that won't break functionality for anyone else, this patch saves the first netdev name we find for the device, and returns that if we fail to find a netdev with the desired phys_port_id. 2017-09-19 Peter Krempa qemu: blockPeek: Enforce buffer filling Documentation states: "'offset' and 'size' represent an area which must lie entirely within the device or file." Enforce the that the buffer lies within fully. 2017-09-19 Peter Krempa qemu: blockPeek: Fix filling of the return buffer Commit 3956af495e broke the blockPeek API since virStorageFileRead allocates a return buffer and fills it with the data, while the API fills a user-provided buffer. This did not get caught by the compiler since the API prototype uses a 'void *'. Fix it by transferring the data from the allocated buffer to the user provided buffer. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1491217 2017-09-19 Andrea Bolognani Revert "travis: Limit git depth to 5 commits" Turns out a build job can be stuck waiting for a macOS worker to become available for a pretty long time: if more than 5 commits have been pushed in the meantime, the clone will be too shallow for the worker to find the commit it's supposed to verify, and the build job will fail. See https://travis-ci.org/libvirt/libvirt/jobs/277244110 for an example of the failure described. This reverts commit 2e975abdc9bbc9e965486e8486cc17a771cdaeb3. Reviewed-by: Daniel P. Berrange 2017-09-19 Andrea Bolognani python: Don't hardcode interpreter path This is particularly useful on operating systems that don't ship Python as part of the base system (eg. FreeBSD) while still working just as well as it did before on Linux. While at it, make it explicit that our scripts are only going to work with Python 2, and remove the usage of unbuffered I/O, which as far as I can tell has no effect on the output files. 2017-09-19 Andrea Bolognani perl: Don't hardcode interpreter path This is particularly useful on operating systems that don't ship Perl as part of the base system (eg. FreeBSD) while still working just as well as it did before on Linux. In one case (src/rpc/genprotocol.pl) the interpreter path was missing altogether. 2017-09-19 Michal Privoznik qemu: Mark graphics ports used on reconnect I don't want to mask the real problem, but one can advocate that we should be marking graphics ports as already in use on qemuProcessReconnect anyway, because we already know that they are taken. 2017-09-19 Ján Tomko configure: fix check for DEVLINK_CMD_ESWITCH_GET Instead of checking for all possible constants that every kernel header with devlink support should have (and defining HAVE_DECL_DEVLINK as 1 if any of them is present due to the way AC_CHECK_DECLS works), only check for DEVLINK_CMD_ESWITCH_GET. This is the name of the constant since kernel 4.11. Between 4.8 and 4.11, the now deprecated spelling DEVLINK_CMD_ESWITCH_MODE_GET was used. Assume DEVLINK_ESWITCH_MODE_SWITCHDEV is available, since it was introduced along with the deprecated spelling. 2017-09-19 John Ferlan storage: Use virStoragePoolObjDefUseNewDef Use the new accessor API for storage_driver. storage: Use virStoragePoolObj{Get|Incr|Decr}Asyncjobs Use the new accessor APIs for storage_driver. storage: Internally represent @autostart as bool Since it's been used that way anyway, let's just convert it to a bool and only make the external representation be an int. storage: Use virStoragePoolObj{Is|Set}Autostart Use the new accessor APIs for storage_driver and test_driver. storage: Use virStoragePoolObj{Is|Set}Active Use the new accessor APIs for storage_driver, test_driver, and gluster backend. storage: Use virStoragePoolObjGetAutostartLink Use the new accessor API for storage_driver. storage: Use virStoragePoolObj{Get|Set}ConfigFile Use the new accessor APIs for storage_driver and test_driver. 2017-09-19 John Ferlan storage: Introduce APIs to search/scan storage pool volumes list Introduce virStoragePoolObjForEachVolume to scan each volume calling the passed callback function until all volumes have been processed in the storage pool volume list, unless the callback function returns an error. Introduce virStoragePoolObjSearchVolume to search each volume calling the passed callback function until it returns true indicating that the desired volume was found. 2017-09-19 John Ferlan storage: Introduce storage volume add, delete, count APIs Create/use virStoragePoolObjAddVol in order to add volumes onto list. Create/use virStoragePoolObjRemoveVol in order to remove volumes from list. Create/use virStoragePoolObjGetVolumesCount to get count of volumes on list. For the storage driver, the logic alters when the volumes.obj list grows to after we've fetched the volobj. This is an optimization of sorts, but also doesn't "needlessly" grow the volumes.objs list and then just decr the count if the virGetStorageVol fails. 2017-09-19 John Ferlan storage: Fill in storage pool @active properly It's a bool not an int, so use true/false and not 1/0 2017-09-19 John Ferlan storage: Introduce virStoragePoolObjNew Create/use a helper to perform object allocation. Adjust storagevolxml2argvtest.c in order to use the allocator and setting of the obj->def. 2017-09-19 John Ferlan storage: Create accessor API's for virStoragePoolObj In preparation for making a private object, create accessor API's for consumer storage functions to use: virStoragePoolObjGetDef virStoragePoolObjSetDef virStoragePoolObjGetNewDef virStoragePoolObjDefUseNewDef virStoragePoolObjGetConfigFile virStoragePoolObjSetConfigFile virStoragePoolObjGetAutostartLink virStoragePoolObjIsActive virStoragePoolObjSetActive virStoragePoolObjIsAutostart virStoragePoolObjSetAutostart virStoragePoolObjGetAsyncjobs virStoragePoolObjIncrAsyncjobs virStoragePoolObjDecrAsyncjobs 2017-09-19 Erik Skultety virsh: man: Describe the 'create' command a bit more So we refer to the terms 'persistent' and 'transient' across the whole man page, without describing it further, but more importantly, how the create command affects it, i.e. explicitly stating that domain created via the 'create' command are going to be transient or persistent, depending on whether there is an existing persistent domain with a matching and , in which case it will remain persistent, but will run using a one-time configuration, otherwise it's going to be transient and will vanish once destroyed. 2017-09-19 Nikolay Shirokovskiy iohelper: reduce zero-out in align case We only need to zero-out bytes that will be written. May be we even don't need to zero-out at all because of immediate truncate. iohelper: simplify last direct write alignment Make alignment of last direct write more straightforward. Using additionally two flags 'end' and 'shortRead' looks complicated. iohelper: drop unused operation length limit 2017-09-18 Guido Günther apparmor: cater for new AAVMF image location Things moved again, sigh. Reviewed-By: Jamie Strandboge Michal Privoznik 2017-09-18 Guido Günther apparmor: add attach_disconnected Otherwise we fail to reconnect to /dev/net/tun opened by libvirtd like [ 8144.507756] audit: type=1400 audit(1505488162.386:38069121): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 profile="libvirt-5dfcc8a7-b79a-4fa9-a41f-f6271651934c" name="dev/net/tun" pid=9607 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=117 ouid=0 Reviewed-By: Jamie Strandboge Acked-By: Michal Privoznik 2017-09-18 Jiri Denemark cpu: Add new Skylake-Server CPU model Available since QEMU 2.10.0 (specifically commit v2.9.0-2233-g53f9a6f45f). Reviewed-by: Pavel Hrdina 2017-09-18 Jiri Denemark cpu: Add clwb/pcommit CPU features The features were added to QEMU by commit v2.4.0-1690-gf7fda28094 as Skylake Server features. Reviewed-by: Pavel Hrdina 2017-09-18 Jiri Denemark tests: Add CPUID data for Intel(R) Xeon(R) Gold 6148 CPU Reviewed-by: Pavel Hrdina 2017-09-18 John Ferlan docs: Update news.xml to describe switchdev offloading 2017-09-18 Edan David nodedev: add switchdev to NIC capabilities Adding functionality to libvirt that will allow querying the interface for the availability of switchdev Offloading NIC capabilities. The switchdev mode was introduced in kernel 4.8, the iproute2-devlink command to retrieve the switchdev NIC feature with command example: devlink dev eswitch show pci/0000:03:00.0 This feature is needed for Openstack so we can do a scheduling decision if the NIC is in Hardware Offload (switchdev) or regular SR-IOV (legacy) mode. And select the appropriate hypervisors with the requested capability see [1]. [1] - https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/enable-sriov-nic-features.html Reviewed-by: Laine Stump Reviewed-by: John Ferlan 2017-09-18 Michal Privoznik qemu: Implement usernet address https://bugzilla.redhat.com/show_bug.cgi?id=1075520 Apart from generic checks, we need to constrain netmask/prefix length a bit. Thing is, with current implementation QEMU needs to be able to 'assign' some IP addresses to the virtual network. For instance, the default gateway is at x.x.x.2, dns is at x.x.x.3, the default DHCP range is x.x.x.15-x.x.x.30. Since we don't expose these settings yet, it's safer to require shorter prefix to have room for the defaults. Reviewed-by: laine@laine.org 2017-09-18 Michal Privoznik conf: Allow usernet to have an address https://bugzilla.redhat.com/show_bug.cgi?id=1075520 Currently, all that users can specify for an interface type of 'user' is the common attributes: PCI address, NIC model (and that's basically it). However, some need to configure other address range than the default one. Reviewed-by: laine@laine.org 2017-09-18 Jiri Denemark cpu_x86: Implement virCPUValidateFeatures The function checks whether all CPU features used in a CPU definition are specified in cpu_map.xml. https://bugzilla.redhat.com/show_bug.cgi?id=1460086 2017-09-18 Jiri Denemark cpu_s390: Implement virCPUValidateFeatures Only feature policy is checked on s390, which was previously done in virCPUUpdate, but that's not the correct place for the check once we have virCPUValidateFeatures. qemu: Validate guest CPU features before starting a domain cpu: Introduce virCPUValidateFeatures This new API may be used to check whether all features used in a CPU definition are valid (e.g., libvirt knows their name, their policy is supported, etc.). Leaving this API unimplemented in an arch subdriver means libvirt does not restrict CPU features usable on the associated architectures. 2017-09-18 Jiri Denemark qemu: Filter CPU features returned by qemuConnectBaselineCPU The host CPU definitions reported in the capabilities XML may contain CPU features unknown to QEMU, but the result of virConnectBaselineCPU is supposed to be directly usable as a guest CPU definition and thus it should only contain features QEMU knows about. https://bugzilla.redhat.com/show_bug.cgi?id=1450317 2017-09-18 Jiri Denemark qemu: Publish virQEMUCapsCPUFilterFeatures qemu: Pass virArch * to virQEMUCapsCPUFilterFeatures The filter only needs to know the CPU architecture. Passing virQEMUCapsPtr as opaque is a bit overkill. cpu: Drop cpuBaselineXML The implementation of virConnectBaselineCPU may be different for each hypervisor. Thus it shouldn't really be implmented in the cpu code. cpu: Don't log CPU models in cpuBaselineXML They are logged in cpuBaseline anyway. cpu: Use virCPUDefListParse in cpuBaselineXML cpu_conf: Introduce virCPUDefList{Parse,Free} For parsing a list of CPU XMLs into a NULL-terminated list of CPU defs. 2017-09-18 Jiri Denemark virsh: Enhance documentation of --rdma-pin-all option https://bugzilla.redhat.com/show_bug.cgi?id=1373783 Reviewed-by: Pavel Hrdina 2017-09-15 Cole Robinson spec: Fix %license compat on RHEL6 The macro needs to be defined closer to its usage 2017-09-15 Cole Robinson spec: Use %license when available This is required by the fedora packaging guidelines: https://fedoraproject.org/wiki/Packaging:LicensingGuidelines This macro isn't available on stock RHEL6 so provide a backcompat definition https://bugzilla.redhat.com/show_bug.cgi?id=1483293 Reported-by: Ville Skyttä 2017-09-15 Ville Skyttä spec: Own %{_libdir}/libvirt{,/connection-driver} dirs Owning all created directories is a requirement of the Fedora packaging guidelines https://bugzilla.redhat.com/show_bug.cgi?id=1483293 2017-09-15 John Ferlan qemu: Fix return check on virHashAddEntry call Luckily it only returns 0 or -1 2017-09-15 John Ferlan conf: Use virXMLFormatElement to format disk source network Commit id 'e02ff020cac' neglected to use the attrBuf and childBuf in the virDomainDiskSourceFormatNetwork call. So make the necessary alterations to allow usage. 2017-09-15 John Ferlan conf: Move encryption validation Rather than checking during XML processing, move the check for valid into virDomainDiskDefParseValidate and alter the text of the message slightly to be a bit more correct. conf: Add invalid domain disk encryption test Add a test to prove checking for invalid luks disk formatting check. The error message will indicate supplying the element is unnecessary. 2017-09-15 John Ferlan conf: Move authdef validation Rather than checking during XML processing, move the checks for correct and valid auth into virDomainDiskDefParseValidate. This will introduce virDomainDiskSourceDefParseAuthValidate to validate that the authdef stored for the virStorageSource is valid. This can then be expanded to service backingStore sources as well. Alter the message text slightly as well to distinguish between an unknown name and an incorrectly used name. Since type is not a mandatory field, add the NULLSTR() around the output of the unknown error. NB, a config using unknown formatting would fail virschematest since it only accepts 'iscsi' and 'ceph' as "valid" types. 2017-09-15 John Ferlan conf: Add invalid secrettype checks Add a couple of tests to "validate" checks in domain_conf that either a missing secrettype (CONFIG_UNSUPPORTED) or an mismatched secrettype of ceph for an iSCSI disk (INTERNAL_ERROR) will cause a parsing error. 2017-09-15 John Ferlan docs: Remove unnecessary example for iscsi disk type='volume' Alter the example to remove the from: and The reality is, it's not even used. For a the authdef from the storage source pool will supercede whatever is in the definition during virStorageTranslateDiskSourcePool processing. In fact, if the pool doesn't have/need authentication, then the authdef would be removed anyway as the storage pool would be handling things. The "proof" for this is in the adjustment to the test to add an for a disk. The resulting .args file won't add what normally would be added "myname:encodedpassword@" prior to the hostname in the IQN (e.g. iscsi://myname:encodedpassword@iscsi.example.org:3260/... 2017-09-15 Peter Krempa qemu: Restore errors when rolling back disk image state Some operations done to rollback disk image labelling and locking might overwrite (or clear) the actual error. Remember the original error when tearing down disk access so that it's not obscured. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1461301 2017-09-15 Peter Krempa qemu: hotplug: Use new helpers for storing libvirt errors The helpers allow to simplify restoring original errors in most cases. util: error: Add helpers for saving and restoring of last error Some cleanup paths overwrite a usefull error message with a less useful one and we then try to preserve the original message. The handlers added in this patch will simplify the operations since they are designed right for the purpose. 2017-09-14 Cole Robinson news: document spice rendernode DAC changes Reviewed-by: Andrea Bolognani 2017-09-14 Andrea Bolognani travis: Shuffle sections around Order them more logically and make sure that stuff that doesn't need to be modified frequently if at all, such as the notification settings, are out of the way. Perform other very minor tweaks as well. Reviewed-by: Martin Kletzander 2017-09-14 Andrea Bolognani travis: Install more build dependencies Since configure automatically picks up as many optional dependencies as possible, installing more packages allows us to improve our test coverage. Reviewed-by: Daniel P. Berrange 2017-09-14 Andrea Bolognani travis: Improve test matrix The default distribution is apparently ignored if an explicit test matrix is provided, so we haven't actually been testing the precise plus gcc combo. Reviewed-by: Daniel P. Berrange 2017-09-14 Andrea Bolognani travis: Don't have a separate script definition for macOS Make parts of the build command OS-dependent instead. Reviewed-by: Daniel P. Berrange 2017-09-14 Andrea Bolognani travis: Don't abort build due to -Wvariadic-macros The openwsman header files are at fault here, but precise is entirely unmaintained at this point so the issue will never be fixed. Better to ignore the error and have coverage over the Hyper-V driver than disabling it: if code that would trigger the warning will be added to libvirt, the CentOS CI will catch it. Reviewed-by: Daniel P. Berrange 2017-09-14 Andrea Bolognani travis: Move variables to 'env' section Reviewed-by: Daniel P. Berrange 2017-09-14 Andrea Bolognani travis: Limit git depth to 5 commits We don't need 50 commits for our purposes, so might as well save some bandwidth and possibly some time by making the clone shallower. Reviewed-by: Daniel P. Berrange 2017-09-14 John Ferlan qemu: Add QEMU 2.10 x86_64 the generated capabilities For reference, these were generated by updating a local qemu git repository to the latest upstream, making sure the latest dependencies were met via "dnf builddep qemu" from my sufficiently privileged root account, checking out the v2.10.0 tag, and building in order to generate an "x86_64-softmmu/qemu-system-x86_64" image. Then using a clean libvirt tree updated to master and built, the image was then provided as input: tests/qemucapsprobe /path/to/x86_64-softmmu/qemu-system-x86_64 > \ tests/qemucapabilitiesdata/caps_2.10.0.x86_64.replies With the .replies file in place and the DO_TEST line added and build, then running the following commands: touch tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml VIR_TEST_REGENERATE_OUTPUT=1 ./tests/qemucapabilitiestest to generate tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml and both were added to the commit. Reviewed-by: Pavel Hrdina 2017-09-14 Peter Krempa qemu: caps: Deprecate QEMU_CAPS_BLOCKJOB_SYNC Interestingly, none of the qemus we have caps for supported it ... Reviewed-by: Eric Blake 2017-09-14 Peter Krempa qemu: monitor: Remove support for "legacy" block jobs Drop all the monitor code necessary to do the downstream block jobs. Reviewed-by: Eric Blake 2017-09-14 Peter Krempa qemu: Remove support for legacy block jobs Block job QMP commands with underscores rather than dashes were never released in upstream qemu, (they were added, but modified in the same release [1]), but a certain distro managed to backport the version in the middle. The change also slightly modified semantics for the abort command, which made us have a lot of code which was only ever present in certain downstream distros. Clean the upstream code from the legacy cruft and support only the upstream implementations. [1] See qemu commit v1.0-2176-gdb58f9c060 Reviewed-by: Eric Blake 2017-09-13 John Ferlan qemu: Clean up qemuDomainSecretPrepare No need to pass a @driver parameter since all that's done is deref the @cfg especially since the only caller can just pass an already referenced @cfg. Also, looks like commit id '0298531b' at one time had a different name for the API, so I took the liberty of fixing the comments too since I would already be updating them for the @cfg variable. 2017-09-13 Michal Privoznik virsh.pod: Fix units for dommemstat and domstats The documentation mistakenly states that the unit for returned values is kB (multiple of 1000), while in fact we are returning KiB (multiple of 1024). 2017-09-12 Cole Robinson security: dac: relabel spice rendernode For a logged in user this a path like /dev/dri/renderD128 will have default ownership root:video which won't work for the qemu:qemu user, so we need to chown it. We only do this when mount namespaces are enabled in the qemu driver, so the chown'ing doesn't interfere with other users of the shared render node path https://bugzilla.redhat.com/show_bug.cgi?id=1460804 2017-09-12 Cole Robinson security: add MANAGER_MOUNT_NAMESPACE flag The VIR_SECURITY_MANAGER_MOUNT_NAMESPACE flag informs the DAC driver if mount namespaces are in use for the VM. Will be used for future changes. Wire it up in the qemu driver 2017-09-12 John Ferlan storage: Adjust expected format for Disk startup processing https://bugzilla.redhat.com/show_bug.cgi?id=1464313 If a Disk pool was defined/created using XML that either didn't specify a specific format or specified format type='unknown', then restarting a pool after an initial disk backend build with overwrite would fail after a libvirtd restart for a non-autostarted pool. This is because the persistent pool data is not updated during pool build w/ overwrite processing to have the VIR_STORAGE_POOL_DISK_DOS default format. So in addition to the alteration done during disk build processing, alter the default expectation for disk startup to be DOS if nothing has been defined yet. That will either succeed if the pool had been successfully built previously using the default DOS format or fail with a message indicating the format is something else that does not match the expect format 'dos'. 2017-09-12 John Ferlan qemu: Provide default LUN=0 for iSCSI if not provided https://bugzilla.redhat.com/show_bug.cgi?id=1477880 If the "/#" is missing from the provided iSCSI path, then we need to provide the default LUN of /0; otherwise, QEMU will fail to parse the URL causing a failure to either create the guest or hotplug attach the storage. During post parse, for any iSCSI disk or hostdev, scan the source path looking for the presence of '/', if found, then we can assume the LUN is provided. If not found, alter the input XML to add the "/0". This will cause the generated XML to have the generated value when the domain config is saved after post parse. 2017-09-12 Peter Krempa qemu: blockcopy: Probe image format only with VIR_DOMAIN_BLOCK_COPY_REUSE_EXT Commit 703abf1d7 changed the logic so that we don't attempt to re-create the image if it's a block device. This was done by modifying the 'reuse' variable. Unfortunately after modifying it one of the uses was to infer whether we should probe the disk format. After changes in the commit mentioned above we would attempt the probe if the target of the copy is a block device and the format was not provided explicitly rather than using the format of the disk. Fix it by explicitly checking whether the user requested a reuse of the disk rather than the modified boolean flag. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1490826 2017-09-12 Michal Privoznik qemu: cold-unplug of watchdog https://bugzilla.redhat.com/show_bug.cgi?id=1447169 Again, no special here. 2017-09-12 Michal Privoznik qemu: cold-plug of watchdog https://bugzilla.redhat.com/show_bug.cgi?id=1447169 With this patch users can cold plug a watchdog. Things are pretty simple because a domain can have at most one watchdog device. 2017-09-12 Michal Privoznik qemuBuildHostNetStr: Don't leak buffer If there was an error when constructing the buffer, NULL is returned. The buffer is never freed though. rng: Fix formatting Some elements are offset just one space compared to their parent, some are misaligned completely, and so on. rng: Drop useless If we have with just one value to chose from, it's no choice. 2017-09-11 Guido Günther virnetserver: fix mesage vs message typo qemu_driver: fix existance vs existence typo storagefile: fix defintion vs definition typo 2017-09-11 Jiri Denemark qemu: Don't report failure to destroy a destroyed domain When destroying a domain libvirt marks it internally with a beingDestroyed flag to make sure the qemuDomainDestroyFlags API itself cleans up after the domain rather than letting an uninformed EOF handler do it. However, when the domain is being started at the moment libvirt was asked to destroy it, only the starting thread can properly clean up after the domain and thus it ignores the beingDestroyed flag. Once qemuDomainDestroyFlags finally gets a job, the domain may not be running anymore, which should not be reported as an error if the domain has been starting up. https://bugzilla.redhat.com/show_bug.cgi?id=1445600 Reviewed-by: Martin Kletzander 2017-09-11 Jiri Denemark virsh: migrate --timeout-postcopy requires --postcopy Requesting an automated switch to a post-copy migration (using --timeout-postcopy) without actually enabling post-copy migration (using --postcopy) doesn't really do anything. Let's make this dependency explicit to avoid unexpected behavior. https://bugzilla.redhat.com/show_bug.cgi?id=1455023 2017-09-11 Ján Tomko conf: validate IOMMU interrupt remapping setting This option requires: Report an error in case someone tries to combine it with different ioapic setting. Setting 'eim' on without enabling 'intremap' does not make sense. https://bugzilla.redhat.com/show_bug.cgi?id=1457610 2017-09-11 Ján Tomko tests: merge iommu tests Using intremap without does not work. Merge the tests to avoid a duplicit test once we start validating it. conf: use virXMLFormatElement for Simplify the formatting function even further. conf: use virXMLFormatElement for Use the new helper to simplify the code. This also fixes the bug of not formatting 'eim' in the useless case if it's the only enabled attribute. 2017-09-08 Erik Skultety virsh: man: Document the --validate option for create and define cmds Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1368753 Reviewed-by: Martin Kletzander 2017-09-08 Erik Skultety docs: Update --timeout description in libvirtd's man page Since commit @ae2163f8, only active client connections or running domains are allowed to inhibit daemon shutdown. The man page however wasn't updated appropriately. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1325066 Reviewed-by: Martin Kletzander 2017-09-07 Andrea Bolognani qemu: Clean up qemuBuildControllerDevCommandLine() Add a 'cleanup' label and improve the readability of one of the checks by making it conform to our formatting standard and moving the corresponding comment. Reviewed-by: Laine Stump 2017-09-07 Stefan Berger tpm: Use /dev/null for cancel path if none was found TPM 2 does not implement sysfs files for cancellation of commands. We therefore use /dev/null for the cancel path passed to QEMU. Tested-by: Javier Martinez Canillas Reviewed-by: Marc-André Lureau 2017-09-07 Brijesh Singh cpu: Add new EPYC CPU model Add a new CPU model called 'EPYC' to model processors from AMD EPYC family (which includes EPYC 76xx,75xx,74xx, 73xx and 72xx). The following features bits have been added/removed compare to Opteron_G5 Added: monitor, movbe, rdrand, mmxext, ffxsr, rdtscp, cr8legacy, osvw, fsgsbase, bmi1, avx2, smep, bmi2, rdseed, adx, smap, clfshopt, sha xsaveopt, xsavec, xgetbv1, arat Removed: xop, fma4, tbm The patch is depend on EPYC CPU model supported introduced in qemu [1] [1] https://patchwork.kernel.org/patch/9902205/ Cc: Tom Lendacky Reviewed-by: Pavel Hrdina 2017-09-07 Jiri Denemark tests: Add CPUID data for AMD EPYC 7601 32-Core Processor Reviewed-by: Pavel Hrdina tests: Add CPUID data for AMD Ryzen 7 1800X Eight-Core Processor Reviewed-by: Pavel Hrdina 2017-09-07 Nikolay Shirokovskiy qemu: migration: don't expose incomplete job as complete In case of real migration (not migrating to file on save, dump etc) migration info is not complete at time qemu finishes migration in normal (non postcopy) mode. We need to update disks stats, downtime info etc. Thus let's not expose this job status as completed. To archive this let's set status to 'qemu completed' after qemu reports migration is finished. It is not visible as complete job to clients. Cookie code on confirm phase will finally turn job into completed. As we don't need more things to do when migrating to file status is set to 'completed' as before in this case. 2017-09-07 Nikolay Shirokovskiy qemu: migrate: add mirror stats to migration stats When getting job info in case mirror does not reach ready phase fetch mirror stats from qemu. Otherwise mirror stats are already saved in current job. qemu: always get job condition on getting job stats Looks like it is more simple to drop this optimization as we are going to add getting disks stats during migration via quering qemu process and checking if we have to acquire job condition becomes more complicate. qemu: introduce migrating job status Instead of checking stat.status let's set status to migrating as soon as migrate command is send (waiting for completion is a good place too). 2017-09-07 Nikolay Shirokovskiy qemu: start all async job with job status active Setting status to none has little value - getting job status will not return even elapsed time. After this patch getting job stats stays correct in a sence it will not fetch migration stats because it consults stats.status before doing the fetch. 2017-09-07 Nikolay Shirokovskiy qemu: fail querying destination migration statistics always Querying destination migration statistics may result in getting a failure or getting a elapsed time value depending on stats.status value which is odd. Instead let's always fail. Clients should be ready to handle this as currently getting failure period can be considerable. qemu: simplify getting completed job stats 2017-09-07 Nikolay Shirokovskiy qemu: refactor fetching migration stats qemuMigrationFetchJobStatus is rather inconvinient. Some of its callers don't need status to be updated, some don't need to update elapsed time right away. So let's update status or elapsed time in callers instead. This patch drops updating job status on getting job stats by client. This way we will not provide status 'completed' while it is not yet updated by migration routine. 2017-09-07 Nikolay Shirokovskiy qemu: drop excessive zero-out in qemuMigrationFetchJobStatus qemuMonitorGetMigrationStats will do it for us anyway. 2017-09-07 Nikolay Shirokovskiy qemu: drop QEMU_MIGRATION_COMPLETED_UPDATE_STATS This way we get stats only in one place. The former code waits for complete/postcopy status basically and don't need to mess with stats. The patch drops raising an error on stats updates failure. This does not make much sense anyway. 2017-09-07 Nikolay Shirokovskiy qemu: introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY Let's introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY state for job.current->status instead of checking job.current->stats.status. The latter can be changed when fetching migration statistics. Moving state function from the variable and leave only store function seems more managable. This patch removes all state checking usage of stats except for qemuDomainGetJobStatsInternal. This place will be handled separately. 2017-09-07 Nikolay Shirokovskiy qemu: introduce qemu domain job status This patch simply switches code from using VIR_DOMAIN_JOB_* to introduced QEMU_DOMAIN_JOB_STATUS_*. Later this gives us freedom to introduce states for postcopy and mirroring phases. qemu: drop code for VIR_DOMAIN_JOB_BOUNDED and timeRemaining qemu driver does not have VIR_DOMAIN_JOB_BOUNDED jobs and timeRemaining is always 0. 2017-09-06 Andrea Bolognani travis: Install gettext msgmerge(1) and friends are required to build libvirt, so the corresponding package should be installed in the Travis worker. travis: Sort build dependencies Keeping the list of build dependencies sorted alphabetically makes it way easier to visually scan it for issues. 2017-09-06 Michal Privoznik conf: Validate device on update-device https://bugzilla.redhat.com/show_bug.cgi?id=1439991 Whenever a device is being updated via virDomainUpdateDeviceFlags() API, we parse the device XML and ideally run some generic checks to validate the configuration (e.g. if device defines per-device boot order but the domain has os/boot element already). Well, that's the theory - due to a missing check we've jumped early from that check function. Reviewed-by: Erik Skultety 2017-09-06 Andrea Bolognani m4: Disable -Wdisabled-optimization After b4f7793ce269, qemuxml2xmltest has apparently become big enough to trigger a compilation error when using --enable-test-coverage on aarch64: CC qemuxml2xmltest.o qemuxml2xmltest.c: In function 'mymain': qemuxml2xmltest.c:1216:1: error: const/copy propagation disabled: 4361 basic blocks and 99285 registers [-Werror=disabled-optimization] } ^ qemuxml2xmltest.c:1216:1: error: PRE disabled: 4361 basic blocks and 99285 registers [-Werror=disabled-optimization] qemuxml2xmltest.c:1216:1: error: const/copy propagation disabled: 4361 basic blocks and 99285 registers [-Werror=disabled-optimization] qemuxml2xmltest.c:1216:1: error: const/copy propagation disabled: 4361 basic blocks and 99285 registers [-Werror=disabled-optimization] However, as the GCC documentation states, this warning is not really caused by issues in our code, so it makes sense to disable it. Reviewed-by: Daniel P. Berrange 2017-09-05 John Ferlan qemu: Remove unused params from qemuDomainDeviceDefValidate Neither @cfg nor (now) @driver is used in the API, so remove them and mark @opaque as UNUSED. NB: Commit id 'fa3c558596' dropped the unused @qemuCaps which was the last consumer of @driver other than @cfg, but even @cfg was never used even in the original implementation from commit id 'd987f63a'. 2017-09-05 Cole Robinson qemu: Default to video type=virtio for machvirt arm/aarch64 -M virt on KVM doesn't and will never work with standard VGA card emulation. The recommended method is to use type=virtio, so let's make it the default for video devices without an explicit type set by the user. https://bugzilla.redhat.com/show_bug.cgi?id=1404112 2017-09-05 Cole Robinson qemu: Set default video type in qemu PostParse And not generic domain_conf code. We will need qemu private functions in a bit. conf: domain: move video type validation to DeviceDefValidate This allows drivers to set their own default. But if a driver neglects to fill one in, we still error like we previously would at parse time. conf: domain: add VIDEO_TYPE_DEFAULT Will be needed for future patches to pull the default video type setting out of XML parsing routines. 2017-09-05 Erik Skultety maint: Fix incorrect parenthesis placement causing true/false assignment There were a few places in our code where the following pattern in 'if' condition occurred: if ((foo = bar() < 0)) do something; This patch adjusts the conditions to the expected format: if ((foo = bar()) < 0) do something; Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1488192 Reviewed-by: Martin Kletzander 2017-09-04 Andrea Bolognani Makefile.nonreentrant: Rebuild against Fedora 26 According to the comments in the file and the git history, the list of forbidden symbols was originally built against Fedora 9 in 2009 (!) and pretty much never refreshed afterwards. Reviewed-by: Daniel P. Berrange 2017-09-04 Andrea Bolognani docs: Fix typo deamon -> daemon Suggested-by: Martin Kletzander Reviewed-by: Martin Kletzander 2017-09-04 Daniel P. Berrange event: ignore attempts to replace the event loop impl Although not previously explicitly documented, the expectation for the libvirt event loop is that an implementation is registered early in application startup, before calling any libvirt APIs and then run forever after. Replacing a previously registered event loop is not safe & subject to races even if virConnectClose has been called on open handles, due to delayed deregistration of callbacks during conenction close. Reviewed-by: Andrea Bolognani 2017-09-04 Daniel P. Berrange Add libxslt as build requires for mingw RPMs The libxslt package is needed since: commit 94d2d6429d686c5af95115d09c01f3c6bd5ea7c6 Author: Daniel P. Berrange Date: Wed Jul 26 17:40:44 2017 +0100 docs: make xmllint & xsltproc compulsory The native RPM had it already, but mingw build was missing it. 2017-09-04 Michal Privoznik lxcStateInitialize: Don't leak driver's caps Funny thing. So when initializing LXC driver's capabilities, firstly the virLXCDriverGetCapabilities() is called. This creates new capabilities, stores them under driver->caps, ref() them and return them. However, the return value is ignored. Secondly, the function is called yet again and since we have driver->caps set, they are ref()-ed again an returned. So in the end, driver's capabilities have refcount of three when in fact they should have refcount of one. Post-release version bump to 3.8.0 2017-09-04 Daniel Veillard Release of libvirt-3.7.0 * docs/news.xml: update for release * po/*.po*: regenerated 2017-09-04 Richard W.M. Jones vmx: Expose VMware Managed Object Reference (moref) in XML. If you use the VDDK library to access virtual machines remotely, you really need to know the Managed Object Reference ("moref") of the VM. This must be passed each time you connect to the API. For example nbdkit's VDDK plugin requires a moref to be passed to mount up a VM's disk remotely: nbdkit vddk user=root password=+/tmp/rootpw \ server=esxi.example.com thumbprint=xx:xx:xx:... \ vm=moref=2 \ file="[datastore1] Fedora/Fedora.vmdk" Getting the moref is a huge pain. To get some idea of what it is, why it is needed, and how much trouble it is to get it, see: https://blogs.vmware.com/vsphere/2012/02/uniquely-identifying-virtual-machines-in-vsphere-and-vcloud-part-1-overview.html https://blogs.vmware.com/vsphere/2012/02/uniquely-identifying-virtual-machines-in-vsphere-and-vcloud-part-2-technical.html However the moref is available conveniently in the internals of the libvirt VMX driver. This patch exposes it as a custom XML element using the same "vmware:" namespace which was previously used for the datacenterpath (see libvirt commit 636a99058758a044). It appears in the XML like this: Fedora ... ha-datacenter 2 Note that the moref can appear as either a simple ID (for esx:// connections) or as a "vm-" (for vpx:// connections). It should be treated by users as an opaque string. 2017-09-02 John Ferlan docs: Remove from entries due to RNG error commit '96e55048' caused make check failure for virschematest: 1929) Checking ../docs/news.xml against ../news.rng ... libvirt: XML Util error : XML document failed to validate against schema: Unable to validate doc against /home/jferlan/git/libvirt.work/docs/schemas/../news.rng Datatype element summary has child elements Element summary failed to validate content Datatype element summary has child elements Element summary failed to validate content ^[[31m^[[1mFAILED^[[0m That's because elements don't appear to be allowed in the schema. Rather than attempt to fix the schema, figured it was simpler to just remove them and let the schema fix happen later. 2017-09-02 Andrea Bolognani news: Prepare for 3.7.0 release Documents some changes that have slipped through the cracks during the development cycle. Reviewed-by: John Ferlan 2017-09-02 Kothapally Madhu Pavan docs: Document managedsave-edit commands support This patch documents support for managedsave-dumpxml, managedsave-define and managedsave-edit commands. Reviewed-by: Andrea Bolognani 2017-09-01 Peter Krempa news: Add stuff for 3.7.0 2017-09-01 Michal Privoznik virnetdaemon: Don't deadlock when talking to D-Bus https://bugzilla.redhat.com/show_bug.cgi?id=1487322 In ace45e67abbd I tried to fix a problem that we get the reply to a D-Bus call while we were sleeping. In that case the callback was never set. So I changed the code that the callback is called directly in this case. However, I hadn't realized that since the callback is called out of order it locks the virNetDaemon. Exactly the very same virNetDaemon object that we are dealing with right now and that we have locked already (in virNetDaemonAddShutdownInhibition()) 2017-09-01 Nikolay Shirokovskiy qemu: handle -1 for pid in qemuDomainGetMachineName We call qemuDomainGetMachineName on domain start. On first start (after daemon start) pid is 0 and virSystemdGetMachineNameByPID don't get called. But after domain shutting down pid became -1 so on next start virSystemdGetMachineNameByPID is called and returned an error. Error is ignored so it is not critical. But at least on my system (systemd-219 with extra patches) systemd-machined is crashed on this request. This behaviour is triggered by eaf2c9f89. Reviewed-by: Daniel P. Berrange 2017-09-01 Michal Privoznik qemu: Forbid rx/tx_queue_size change explicitly https://bugzilla.redhat.com/show_bug.cgi?id=1484230 When updating a virtio enabled vNIC and trying to change either of rx_queue_size or tx_queue_size success is reported although no operation is actually performed. Moreover, there's no way how to change these on the fly. This is due to way we check for changes: explicitly for each struct member. Therefore it's easy to miss one. 2017-08-31 Shuang He docs: Add entry for ZStack to apps page 2017-08-31 Pavel Hrdina news: add an entry for chardev reconnect feature 2017-08-30 John Ferlan storage: Use virStorageBackendRefreshVolTargetUpdate after wipeVol https://bugzilla.redhat.com/show_bug.cgi?id=1437797 Rather than using refreshVol which essentially only updates the allocation, capacity, and permissions for the volume, but not the format which does get updated in a pool refresh - let's use the same helper that pool refresh uses in order to update the volume target. 2017-08-30 John Ferlan storage: Introduce virStorageBackendRefreshVolTargetUpdate Create a separate function to handle the volume target update via probe processing. 2017-08-30 Pavel Hrdina qemu: set bind mode for chardev while parsing XML Currently while parsing domain XML we clear the UNIX path if it matches one of the auto-generated paths by libvirt. After that when the guest is started new path is generated but the mode is also changed to "bind". In the real-world use-case the mode should not change, it only happens if a user provides a mode='connect' and path that matches one of the auto-generated path or not provides a path at all. Before *reconnect* feature was introduced there was no issue, but with the new feature we need to make sure that it's used only with "connect" mode, therefore we need to move the mode change into parsing in order to have a proper error reported by validation code. 2017-08-30 Pavel Hrdina tests: don't use unix socket path that matches auto-generated path The test was introduced by 60135b22db6d. The auto-generated path is removed by post-parse callback which also changes the mode from "connect" to "bind" since the auto-generated path makes sense only for "bind" mode. 2017-08-30 Pavel Hrdina conf: add reconnect to virDomainChrSourceDef(Copy|IsEqual) Missed by 9aa72a6dd5b3. tests: remove unused file Introduced by 95fd63b1700d. 2017-08-30 Daniel P. Berrange Fix TLS test suites with gnutls 3.6.0 With gnutls 3.6.0, SHA1 is no longer accepted for certificate signatures. We must usw SHA256 instead. 2017-08-30 Kothapally Madhu Pavan doc: Fix docs/news.xml structure 2017-08-30 Martin Kletzander tests: Fix virnetsockettest after SSH command line changes Commit e4cb8500810a changed the way ssh command line is created by adding '--' before the hostname in order to fix a potential security flaw. However it failed to modify the tests, so let's do that. 2017-08-30 Michal Privoznik docs: Document yet another limitation of tx_queue_size https://bugzilla.redhat.com/show_bug.cgi?id=1484234 Turns out, only vhostuser type of interfaces are supported currently. 2017-08-29 Daniel P. Berrange rpc: avoid ssh interpreting malicious hostname as arguments Inspired by the recent GIT / Mercurial security flaws (http://blog.recurity-labs.com/2017-08-10/scm-vulns), consider someone/something manages to feed libvirt a bogus URI such as: virsh -c qemu+ssh://-oProxyCommand=gnome-calculator/system In this case, the hosname "-oProxyCommand=gnome-calculator" will get interpreted as an argument to ssh, not a hostname. Fortunately, due to the set of args we have following the hostname, SSH will then interpret our bit of shell script that runs 'nc' on the remote host as a cipher name, which is clearly invalid. This makes ssh exit during argv parsing and so it never tries to run gnome-calculator. We are lucky this time, but lets be more paranoid, by using '--' to explicitly tell SSH when it has finished seeing command line options. This forces it to interpret "-oProxyCommand=gnome-calculator" as a hostname, and thus see a fail from hostname lookup. 2017-08-29 Martin Kletzander qemu: Also treat directories properly when using namespaces When recreating folders with namespaces, the directory type was not being handled at all. It's not special, we probably just didn't know that that can be used as a volume path as well. The code failed gracefully, but we want to allow that so that we can use in domains again. Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1443434 2017-08-29 Martin Kletzander qemu: Don't mangle the storage format for type='dir' Our backing probing code handles directory file types properly in virStorageFileGetMetadataRecurse(), by that I mean it leaves them alone. However its caller, the virStorageFileGetMetadata() resets the type to raw before probing, without even checking the type. We need to special-case TYPE_DIR in order to achieve desired results. Also, in order to properly test this, we need to stop resetting format of volumes in tests for TYPE_DIR (probably the reason why we didn't catch that and why the test data didn't need to be modified). Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1443434 2017-08-29 Kothapally Madhu Pavan virsh: Implement managedsave-edit command Add a simple virsh command handler which makes use of the new API. virsh: Implement managedsave-dumpxml command Add a simple virsh command handler which makes use of the new API. virsh: Implement managedsave-define command Add a simple virsh command handler which makes use of the new API. qemu: Implement qemuDomainManagedSaveDefineXML This commit adds qemu driver implementation to edit xml configuration of managed save state file of a domain. qemu: Implement qemuDomainManagedSaveGetXMLDesc This commit adds qemu driver implementation to get xml description for managed save state domain. lib: Add API to edit domain's managed save state xml configuration Similar to domainSaveImageDefineXML this commit adds domainManagedSaveDefineXML API which allows to edit domain's managed save state xml configuration. lib: Add API to dump xml configuration of managed save state domain Similar to domainSaveImageGetXMLDesc this commit adds domainManagedSaveGetXMLDesc API which allows to get the xml of managed save state domain. 2017-08-29 Michal Privoznik qemu: Honour https://bugzilla.redhat.com/show_bug.cgi?id=1476866 For some reason, we completely ignore setting for domains. The implementation is simply not there. It never was. 2017-08-29 Michal Privoznik qemuDomainUndefineFlags: Grab QEMU_JOB_MODIFY This API is definitely modifying state of @vm. Therefore it should grab a job. Reviewed-by: John Ferlan 2017-08-29 Michal Privoznik qemu: Introduce and use qemuDomainRemoveInactiveJob At some places we either already have synchronous job or we just released it. Also, some APIs might want to use this code without having to release their job. Anyway, the job acquire code is moved out to qemuDomainRemoveInactiveJob so that qemuDomainRemoveInactive does just what it promises. Reviewed-by: John Ferlan 2017-08-29 Martin Kletzander conf: Clean up and report error in virDomainGenerateMachineName conf: Clean up and report error in virDomainCapsFormat util: Use virBufferCheckError to its full potential. We can now check for the error and not care about the return value as it will be properly handled in virBufferContentAndReset() anyway. util: Umark virBufferCheckErrorInternal as ATTRIBUTE_RETURN_CHECK The function is useful even without using the return value. And if needed, the return value can be obtained by other calls as well. The potential for clean-up can be seen in the following patch. docs: Update news with domain name bug fixes 2017-08-29 Martin Kletzander qemu: Use short domain name in qemuDomainGetPreservedMountPath Otherwise longer domain names might generate paths that are too long to be created. This follows what other parts of the code do as well. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1453194 2017-08-29 Martin Kletzander conf: Properly truncate wide character names in virDomainObjGetShortName We always truncated the name at 20 bytes instead of characters. In case 20 bytes were in the middle of a multi-byte character, then the string became invalid and various parts of the code would error out (e.g. XML parsing of that string). Let's instead properly truncate it after 20 characters instead. We cannot test this in our test suite because we would need to know what locales are installed on the system where the tests are ran and if there is supported one (most probably there will be, but we cannot be 100% sure), we could initialize gettext in qemuxml2argvtest, but there would still be a chance of getting two different (both valid, though) results. In order to test this it is enough to start a machine with a name for which trimming it after 20 bytes would create invalid sequence (e.g. 1234567890123456789č where č is any multi-byte character). Then start the domain and restart libvirtd. The domain would disappear because such illegal sequence will not go through the XML parser. And that's not a bug of the parser, it should not be in the XML in the first place, but since we don't use any sophisticated formatter, just mash some strings together, the formatting succeeds. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1448766 2017-08-29 Martin Kletzander virsh: Honour --readonly with cmdConnect and no name Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1436042 docs: Update news with virtio-vga/gpu's max_outputs= parameter qemu: Add support for virtio-vga/gpu's max_outputs= parameter Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1449712 qemu: Add capabilities for virtio-vga/gpu's max_outputs= parameter 2017-08-29 Pavel Hrdina tests: add qemu chardev source reconnect tests 2017-08-29 Pavel Hrdina qemu: implement chardev source reconnect The reconnect attribute for chardev devices in QEMU is used to configure the reconnect timeout in seconds. Setting '0' value disables the reconnect functionality thus we don't allow to set '0' for QEMU. To disable the reconnect user should use . Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1254971 2017-08-29 Pavel Hrdina tests: add generic xml chardev source reconnect tests conf: introduce reconnect element for chardev source qemu: introduce QEMU_CAPS_CHARDEV_RECONNECT 2017-08-28 John Ferlan network: Fix virNetworkObjBridgeInUse return type Rather than an int, it returns a bool - so let's define it that way 2017-08-28 John Ferlan Revert "interface: Consume @def in virInterfaceObjNew" This reverts commit 92840eb3a7e47cdf761e52afccc41d2a35327fbd. More recent reviews/changes don't have the vir*ObjNew APIs consuming the @def, so remove from Interface as well. Changes needed to also deal with conflicts from commit id '46f5eca4'. 2017-08-28 Eric Blake maint: correct quoting for gl_WARN_ADD 3 out of 4 uses of gl_WARN_ADD() were incorrectly adding "" around the argument, which in turn resulted in the argument being used unquoted (configure had gl_positive=""-fstack-protector-all"", rather than the intended gl_positive="-fstack-protector-all"). Reviewed-by: Andrea Bolognani 2017-08-28 Erik Skultety daemon: logging: Fix --verbose option being ignored by the daemon Commit 94c465d0 refactored the logging setup phase but introduced an issue, where the daemon ignores verbose mode when there are no outputs defined and the default must be used. The problem is that the default output was determined too early, thus ignoring the potential '--verbose' option taking effect. This patch postpones the creation of the default output to the very last moment when nothing else can change. Since the default output is only created during the init phase, it's safe to leave the pointer as NULL for a while, but it will be set eventually, thus not affecting runtime. Patch also adjusts both the other daemons. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1442947 2017-08-28 Andrea Bolognani qemu: Handle host devices not being available better We can't retrieve the isolation group of a device that's not present in the system. However, it's very common for VFs to be created late in the boot, so they might not be present yet when libvirtd starts, which would cause the guests using them to disappear. Moreover, for other architectures and even ppc64 before isolation groups were introduced, it's considered perfectly fine to configure a guest to use a device that's not yet (or no longer) available to the host, with the obvious caveat that such a guest won't be able to start before the device is available. In order to be consistent, when a device's isolation group can't be determined fall back to not isolating it rather than erroring out or, worse, making the guest disappear. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1484254 2017-08-28 Pavel Hrdina conf: don't close the source element inside different function While formatting disk or chardev element they both uses virDomainDiskSourceDefFormatSeclabel() function which also closes the source element. This is not extendable. Use the new virXMLFormatElement() to properly format the source element with possible child elements. As a side effect it fixes a bug in disk source formatting. Reviewed-by: John Ferlan 2017-08-28 Pavel Hrdina util: introduce virXMLFormatElement helper This helper allows you to better structurize the code if some element may or may not contains attributes and/or child elements. Reviewed-by: John Ferlan 2017-08-28 Pavel Hrdina util: introduce virBufferSetChildIndent macro Reviewed-by: John Ferlan 2017-08-28 Erik Skultety maint: Update to latest gnulib This pulls in, among other new things, vc-list-files fix to make syntax-check work with git worktrees. 2017-08-27 Cole Robinson conf: add virDomainVideoDefNew To handle setting a default heads value. Convert callers that were doing it by hand Reviewed-by: John Ferlan 2017-08-27 Cole Robinson qemu: domain: Move some validation out of DeviceDefPostParse And into DeviceDefValidate which is the expected place Reviewed-by: John Ferlan 2017-08-27 Cole Robinson qemu: parse: drop redundant video config The ram/vram = 0 bits aren't needed, and PostParse will fill in the needed QXL default Reviewed-by: John Ferlan 2017-08-27 Cole Robinson qemu: Remove remnants of xenner support Both of these are dead code: qemu_command.c explicitly rejects VIRT_XEN earlier in the call chain, and qemu_parse_command.c will never set VIRT_XEN anymore Reviewed-by: John Ferlan 2017-08-26 Scott Garfinkle docs: document migrate-getmaxdowntime support virsh: Add support for virDomainMigrateGetMaxDowntime Implement a migrate-getmaxdowntime command to complement migrate-setmaxdowntime. qemu: Implement virDomainMigrateGetMaxDowntime Add code to support querying maximum allowable downtime during live migration. Add virDomainMigrateGetMaxDowntime public API Add virDomainMigrateGetMaxDowntime to support querying maximum allowable downtime during live migration. 2017-08-25 Andrea Bolognani qemu: Report error on failure to set isolation group This is more user-friendly because the error will be displayed directly instead of being buried in the log. Reviewed-by: Pavel Hrdina 2017-08-25 Nitesh Konkar virt-host-validate: Fix warning for IOMMU detection on PPC Fix the warning generated on PPC by virt-host-validate for IOMMU. In case of PPC, IOMMU in the host kernel either has it or it's not compiled in. The /sys/kernel/iommu_groups check is good enough to verify if it was compiled with the kernel or not. Modify the error message when "if (sb.st_nlink <= 2)" to indicate what the problem would be since there would be no @bootarg. 2017-08-24 Erik Skultety Revert "maint: Update to latest gnulib" Revert @f8172388c which broke the build on Centos 6, because of old autoconf (< 2.63b) not properly quoting arguments for shell. 2017-08-24 George Dunlap libxl: Avoid a variable named 'stat' Using a variable named 'stat' clashes with the system function 'stat()' causing compiler warnings on some platforms: libxl/libxl_driver.c: In function 'libxlDomainBlockStatsVBD': libxl/libxl_driver.c:5387: error: declaration of 'stat' shadows a global declaration [-Wshadow] /usr/include/sys/stat.h:455: error: shadowed declaration is here [-Wshadow] Reviewed-by: Andrea Bolognani 2017-08-24 Erik Skultety maint: Update to latest gnulib This pulls in, among other new things, vc-list-files fix to make syntax-check work with git worktrees. 2017-08-24 Ján Tomko vmx: do not treat controllers as implicit devices When parsing the config, we look for the SCSI controllers one by one, remembering their models, then let virDomainDefAddImplicitDevices add them if any SCSI disk is using them. Since these controllers are not really implicit (they are present in the source config), add them explicitly. This patch maintains the behavior of not adding a controller if it was present in the config, but no disk was using it. This also resolves the memory leak of virVMXParseConfig overwriting the video device added by calling virDomainDefAddImplicitDevices before the parsing is finished. Reported-by: Michal Privoznik 2017-08-24 Nikolay Shirokovskiy vz: build fix 6e6faf6d changed vzDomObjAlloc signature in source but not in header file. 2017-08-24 Ján Tomko conf: check address type for USB hostdevs https://bugzilla.redhat.com/show_bug.cgi?id=1225339 2017-08-24 Ján Tomko conf: move hostdev address validation to virDomainHostdevDefValidate For selected hostdev types, we validate that the address type matches the subsystem type when parsing the XML. Move it to the validation phase, to allow extending the checks to other subsystem types without making existing domains disappear. 2017-08-23 John Ferlan tests: Fix leak in securityselinuxtest If we jump to the error: label and @secbuf is allocated, then it's not free'd at all. Found by Coverity 2017-08-23 Nikolay Shirokovskiy vz: support disabled items in vz boot order At the time the check was written virtuozzo did not use disabled items in boot order configuration. Boot items were always enabled. Now they can be disabled as well. Supporting such items is easy - they just should be ignored. 2017-08-23 Andrea Bolognani docs: Define anchors correctly in pci-hotplug HTML5 obsoletes the 'name' attribute in favor of 'id', and our TOC generator apparently follows the recommendation to the letter, resulting in a broken TOC if you use the old-school attribute. 2017-08-22 Ján Tomko conf: do not count per-device boot elements when parsing When parsing bootable devices, we maintain a bitmap of used elements. Use it in the post-parse function to figure out whether the user tried to mix per-device and per-domain boot elements. This removes the need to count them twice. 2017-08-22 Ján Tomko conf: create a thin wrapper above virDomainDefPostParse Rename the original function to virDomainDefPostParseInternal to allow adding arguments that will be only used by the internal version. conf: rename virDomain*PostParseInternal to virDomain*PostParseCommon These functions contain the post-parse steps common for all drivers. Rename it to use the 'Common' prefix, instead of the vagueness of 'Internal', leaving 'Internal' available for other vague uses. 2017-08-22 Pavel Hrdina conf: move chardev validation into virDomainDeviceDefValidateInternal conf: separate PTY chardev source parsing There is no reason why to share the same code for PTY and other file based chardev source types. conf: move FILE chardev source parsing to separate function conf: assign parsed strings directly into chardev source definition Since the source element is parsed only once for these type of character devices we don't have to use temporary variable and check whether the variable was already set. conf: move UNIX chardev source parsing to separate function conf: move UDP chardev source parsing to separate function The extra check whether (connect|bind)(Host|Service) was set is required because for UDP chardev there can be two source elements. Without the check there could be a memory leak. conf: move TCP chardev source parsing to separate function conf: move mode parsing of chardev source to separate function conf: move chardev log parsing to separate function conf: move chardev protocol parsing to separate function In order to ensure that the default protocol is RAW, explicitly assigning VIR_DOMAIN_CHR_TCP_PROTOCOL_RAW = 0. conf: error out for multiple protocol elements while parsing chardev Remove check whether a variable was already set because the element is parsed only once now. conf: error out for multiple log elements while parsing chardev Remove check whether a variable was already set because the element is parsed only once now. 2017-08-22 Pavel Hrdina conf: error out for multiple source elements while parsing chardev Currently we accept and correctly parse this chardev XML: ... ... The parsed formatted XML is: ... ... That behavior is super wrong and should not be allowed. If you notice the current parse takes the first found attribute and uses that value, so for example from the "" only the "host" attribute is used. It works the same way for all possible attributes that we are able to parse for source element. This patch enforces providing only one source element for all character devices, only for UDP type we allow to provide two source elements since you can specify both modes. 2017-08-22 Pavel Hrdina conf: switch from while to for loop for chardev parsing This removes one level of indentation. tests: introduce genericxml test for UNIX chardev tests: introduce genericxml test for UDP chardev tests: introduce genericxml test for TCP chardev 2017-08-22 Ján Tomko conf: useserial: drop useless check for serial devices Since its introduction in commit 874e65aa, if someone requests: we report an error if we cannot successfully count the number of serial devices via an XPath query. Instead of fixing the check (and moving it to the validation phase, to prevent existing domains from disappearing), drop it completely. For QEMU, the number of serials is checked when building the command line. 2017-08-21 Jim Fehlig Don't autogenerate seclabels of type 'none' When security drivers are active but confinement is not enabled, there is no need to autogenerate elements when starting a domain def that contains no elements. In fact, autogenerating the elements can result in needless save/restore and migration failures when the security driver is not active on the restore/migration target. This patch changes the virSecurityManagerGenLabel function in src/security_manager.c to only autogenerate a element if none is already defined for the domain *and* default confinement is enabled. Otherwise the needless autogeneration is skipped. Resolves: https://bugzilla.opensuse.org/show_bug.cgi?id=1051017 2017-08-21 Jim Fehlig Fix building domain def in securityselinuxtest The virDomainDef created by testBuildDomainDef in securityselinuxtest adds a seclabel but does not increment nseclabels. Also, it should populate seclabel->model with 'selinux'. While at it, use the secdef itself to populate values instead of the indirection through def->seclabels[0]. 2017-08-21 Lily Zhu tools: Fix docs in libvirt-guests configuration file It was not entirely clear that PARALLEL_SHUTDOWN setting is applied only when the desired action is "shutdown". 2017-08-21 Andrea Bolognani docs: Improve PCI topology and hotplug guidelines Address some minor flaws in the original document that were pointed out during review. 2017-08-21 Andrea Bolognani conf: Use the correct limit for the number of PHBs I mistakenly thought pSeries guests supported 32 PHBs, but it turns out they only support 31. Validate the target index accordingly. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1479647 2017-08-21 Andrea Bolognani conf: Move target index validation Validation should happen after parsing, so the proper location for it is virDomainControllerDefValidate() rather than virDomainControllerDefParseXML(). tests: Improve target index validation coverage Split one of the existing tests to ensure both configuration errors it contained cause a failure, and introduce a new test case. 2017-08-18 Peter Krempa qemu: Implement postParse callback skipping on config reload Use the new facility which allows to ignore failures in post parse callbacks if they are not fatal so that VM configs are not lost if the emulator binary is missing. If qemuCaps can't be populated on daemon restart skip certain portions of the post parse callbacks during config reload and re-run the callback during VM startup. This fixes VMs vanishing if the emulator binary was broken or uninstalled and libvirtd was restarted. 2017-08-18 Peter Krempa qemu: domain: Don't set default USB model if qemuCaps is missing qemuDomainControllerDefPostParse assigns the default USB controller model when it was not specified by the user. Skip this step if @qemuCaps is missing so that we don't fill wrong data. This will then be fixes by re-running the post parse callback. qemu: domain: Don't return default NIC model if @qemuCaps are missing Return NULL in qemuDomainDefaultNetModel if qemuCaps is missing and the network card model would be determined by the capabilities. qemu: capabilities: Tolerate missing @qemuCaps in virQEMUCapsSupportsGICVersion Report the given GIC version as unsupported if @qemuCapsi is NULL. This will be helpful to run post parse callbacks even if qemu is not currently installed. 2017-08-18 Peter Krempa qemu: capabilities: Tolerate missing @qemuCaps in virQEMUCapsGetCanonicalMachine If qemuCaps are not present, just return the original machine type name. This will help in situations when qemuCaps is not available in the post parse callback. 2017-08-18 Peter Krempa conf: add infrastructure for tolerating certain post parse callback failures Some failures of the post parse callback can be tolerated. This is specifically desired when loading the configs of existing VMs. In such case the post parse callback should not really be modifying anything in the definition. This patch adds a parse flag VIR_DOMAIN_DEF_PARSE_ALLOW_POST_PARSE_FAIL which will allow the callbacks to report non-fatal failures by returning a positive return value. In such case the field 'postParseFailed' in the domain definition is set to true, to notify the drivers that the callback failed and possibly needs to be re-run. 2017-08-18 Peter Krempa conf: Return any non-zero value from virDomainDeviceInfoIterateInternal callback Post parse callbacks will need to be able to signal that they failed non-fatally. This means that we need to return the value returned by the callback without modification. qemu: domain: Don't re-allocate qemuCaps in post parse callbacks The domain post parse callback, domain address callback and the domain device callback (for every single device) would each grab qemuCaps for the current emulator. This is quite wasteful. Use the new callback to do this just once. 2017-08-18 Peter Krempa conf: Add callbacks that allocate per-def private data Some drivers use def-specific private data across callbacks (e.g. qemuCaps in the qemu driver). Currently it's mostly allocated in every single callback. This is rather wasteful, given that every single call to the device callback allocates it. The new callback will allocate the data (if not provided externally) and then use it for the VM, address and device post parse callbacks. 2017-08-18 Peter Krempa qemu: Move assignment of default emulator to the basic post parse callback 2017-08-18 Peter Krempa conf: Add 'basic' post parse callback Add yet another post parse callback, which is executed prior the real one without @parseOpaque. This is meant to set basics before @parseOpaque (in case of the qemu driver qemuCaps) can be allocated. This callback will allow to optimize passing of custom parseOpaque through the callbacks. 2017-08-18 Peter Krempa conf: domainlist: Explicitly report failure to load domain config When dropping a domain report which one was dropped so that it's not necessary to rummage through the logs. 2017-08-18 Pavel Hrdina qemu: don't check whether offline migration is safe Offline migration transfers only the domain definition. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1449715 2017-08-18 Peter Krempa util: string: Introduce virStringHasChars The helper returns true if a string contains any of the given chars. virStringHasControlChars can be reimplemented using that helper. Reviewed-by: Pavel Hrdina 2017-08-17 Erik Skultety nodedev: Introduce udevHandleOneDevice Let this new method handle the device object we obtained from the monitor in order to enhance readability. nodedev: udev: Remove the udevEventHandleCallback on fatal error So we have a sanity check for the udev monitor fd. Theoretically, it could happen that the udev monitor fd changes (due to our own wrongdoing, hence the 'sanity' here) and if that happens it means we are handling an event from a different entity than we think, thus we should remove the handle if someone somewhere somehow hits this hypothetical case. nodedev: mdev: Report an error when mdev path resolution fails It might happen that virFileResolveLinkHelper fails on the lstat system call. virFileResolveLink expects the caller to report an error when it fails, however this wasn't the case for udevProcessMediatedDevice. 2017-08-17 Christian Ehrhardt virt-aa-helper: locking loader/nvram for qemu 2.10 Testing qemu-2.10-rc3 shows issues like: qemu-system-aarch64: -drive file=/home/ubuntu/vm-start-stop/vms/ 7936-0_CODE.fd,if=pflash,format=raw,unit=1: Failed to unlock byte 100 There is an apparmor deny due to qemu now locking those files: apparmor="DENIED" operation="file_lock" [...] name="/home/ubuntu/vm-start-stop/vms/7936-0_CODE.fd" name="/var/lib/uvtool/libvirt/images/kvmguest-artful-normal.qcow" [...] comm="qemu-system-aarch64" requested_mask="k" denied_mask="k" The profile needs to allow locking for loader and nvram files via the locking (k) rule. 2017-08-17 Christian Ehrhardt virt-aa-helper: locking disk files for qemu 2.10 Testing qemu-2.10-rc2 shows issues like: qemu-system-x86_64: -drive file=/var/lib/uvtool/libvirt/images/kvmguest- \ artful-normal.qcow,format=qcow2,if=none,id=drive-virtio-disk0: Failed to lock byte 100 It seems the following qemu commit changed the needs for the backing image rules: (qemu) commit 244a5668106297378391b768e7288eb157616f64 Author: Fam Zheng file-posix: Add image locking to perm operations The block appears as: apparmor="DENIED" operation="file_lock" [...] name="/var/lib/uvtool/libvirt/images/kvmguest-artful-normal.qcow" [...] comm="qemu-system-x86" requested_mask="k" denied_mask="k" With that qemu change in place the rules generated for the image and backing files need the allowance to also lock (k) the files. Disks are added via add_file_path and with this fix rules now get that permission, but no other rules are changed, example: - "/var/lib/uvtool/libvirt/images/kvmguest-artful-normal-a2.qcow" rw, + "/var/lib/uvtool/libvirt/images/kvmguest-artful-normal-a2.qcow" rwk 2017-08-17 Pavel Hrdina conf: use virXMLPropString and virXMLNodeContentString for vcpu parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLNodeContentString for boot options parsing Using XPath here doesn't add any benefit. conf: use virXMLPropStringLimit where it makes sense The XPath call for these cases is more expensive than accessing the XML dom node directly. conf: use virXMLPropString for actual network parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for boot parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for network parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for IOMMU def parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for virDomainVirtioOptionsParseXML XPath is good for random search of elements, not for accessing attributes of one node. util: introduce virXMLNodeContentString It's equivalent of calling virXPathString("string(.)", ctxt) but it doesn't have to use the XPath resolving and parsing. util: introduce virXMLPropStringLimit The virXMLPropStringLimit is an equivalent of virXPathStringLimit which should be preferred if you already have a XML dom node or if you need to parse more than one property. 2017-08-17 Michal Privoznik network: Use self inflating bitmap for class IDs Back in the day when I was implementing QoS for networks there were no self inflating virBitmaps. Only the static ones. Therefore, I had to allocate the whole 8KB of memory in order to keep track of used/unused class IDs. This is rather wasteful because nobody is ever gonna use that much classes (kernel overhead would drastically lower the bandwidth). Anyway, now that we have self inflating bitmaps we can start small and allocate more if there's need for it. Reviewed-by: John Ferlan 2017-08-17 John Ferlan network: Use @nnames instead of @got To make it clearer, let's use @nnames instead of @got for counting the names in the @names array. Keeps things consistent and clear. 2017-08-16 Cole Robinson tests: add qemu x86 kvm 32-on-64 test There's some specific logic in qemuBuildCpuCommandLine to support auto adding -cpu qemu 32 for arch=i686 with an x86_64 qemu binary. Add a test case for it 2017-08-16 John Ferlan storage: Alter @aclfilter to just @filter Rename the variable, recent review requested just use of @filter, so be consistent throughout. secret: Alter @aclfilter to just @filter Rename the variable, recent review requested just use of @filter, so be consistent throughout. 2017-08-16 John Ferlan nwfilter: Alter @aclfilter to just @filter Rename the variable, recent review requested just use of @filter, so be consistent throughout. NB: Also change the virNWFilterPtr to be @nwfilter to not conflict with the renamed variable. 2017-08-16 John Ferlan nodedev: Alter @aclfilter to just @filter Rename the variable, recent review requested just use of @filter, so be consistent throughout. network: Privatize virNetworkObj Move from virnetworkobj.h into virnetworkobj.c network: Modify naming for virNetworkObjList* fetching APIs Use the structure names in the @data setup - makes it easier than going back to find the struct fields to make sure the order of the data is correct. network: Use @maxnames instead of @nnames To be consistent with the API definition, use the @maxnames instead of @nnames when describing/comparing against the maximum names to be provided for the *ConnectList[Defined]Networks APIs. network: Move virObjectRef during AssignDef processing Move the virObjectRef in virNetworkObjAssignDefLocked to after the virHashAddEntry to make it "clearer" why the @ref is being incremented. Upon return from the ObjNew we will have 1 ref on the object already, adding it to the hash table requires the increment. network: Have virNetworkObjNew lock the returned object Forces callers to use the virNetworkObjEndAPI properly. network: Consistent use of @obj for virnetworkobj Consistently use @obj for a virNetworkObjPtr. network: Introduce virNetworkObjIsPersistent In preparation to privatize the virNetworkObj - create an accessor function to get the current @persistent value. Also change the value to a bool rather than an unsigned int (since that's how it's generated anyway). 2017-08-16 John Ferlan network: Introduce virNetworkObj{Is|Set}Active In order to privatize the virNetworkObj create accessors in virnetworkobj in order to handle the get/set of the active value. Also rather than an unsigned int, convert it to a boolean to match other drivers representation and the reality of what it is. 2017-08-16 John Ferlan network: Introduce virNetworkObj{Is|Set}Autostart In preparation for privatizing the virNetworkObj structure, create accessors for the obj->autostart. 2017-08-16 John Ferlan network: Add virNetworkObj Get/Set API's for @def and @newDef In preparation for making the object private, create a couple of API's to get the obj->def & obj->newDef and set the obj->def. While altering networkxml2conftest.c to use the virNetworkObjSetDef API, fix the name of the variable from @dev to @def 2017-08-16 John Ferlan network: Add virNetworkObj Get/Set API's for @floor_sum In preparation for making the object private, create a couple of API's to get/set the obj->floor_sum. network: Introduce virNetworkObjGetClassIdMap In preparation for privatizing virNetworkObj, create accessor function to fetch the @classIdMap. network: Alter virNetworkObj @class_id to be @classIdMap Change the variable name to be a bit more descriptive and less confusing when used with the data.network.actual->class_id. 2017-08-16 John Ferlan network: Add virNetworkObj Get/Set API's for @dnsmasqPid and @radvdPid In preparation for making the object private, create/use a couple of API's to get/set the obj->dnsmasqPid and obj->radvdPid. NB: Since the pid's can sometimes changed based on intervening functions, be sure to always fetch the latest value. 2017-08-16 John Ferlan network: Unconditionally initialize macmap when stopping virtual network Since we can only ever have one reference to obj->macmap, rather than only clearing obj->macmap during virNetworkObjUnrefMacMap (e.g. virtual network from networkShutdownNetwork), let's just unconditionally clear the obj->macmap to ensure that some future change that created it's own reference to obj->macmap wouldn't have that reference disappear if virNetworkObjDispose got called. network: Move macmap mgmt from bridge_driver to virnetworkobj In preparation for having a private virNetworkObj - let's create/move some API's that handle the obj->macmap. The API's will be renamed to have a virNetworkObj prefix to follow conventions and the arguments slightly modified to accept what's necessary to complete their task. network: Move and rename networkMacMgrFileName Move networkMacMgrFileName into src/util/virmacmap.c and rename to virMacMapFileName. We're about to move some more MacMgr processing files into virnetworkobj and it doesn't make sense to have this helper in the driver or in virnetworkobj. 2017-08-16 John Ferlan network: Use consistent naming in bridge_driver for virNetwork objects Use @obj for a virNetworkObjPtr Use @net for a virNetworkPtr 2017-08-16 John Ferlan network: Perform some formatting cleanup in bridge_driver.h Alter prototypes to utilize some more recent guidelines. 2017-08-16 Michal Privoznik libvirtd.conf: Drop max_requests Since its introduction in f61341173bdaa2e0 it was never implemented nor there are plans to implement it. Drop it. 2017-08-15 John Ferlan qemu: Fix bug assuming usage of default UUID for certificate passphrase If an environment specific _tls_x509_cert_dir is provided, then do not VIR_STRDUP the defaultTLSx509secretUUID as that would be for the "default" environment and not the vnc, spice, chardev, or migrate environments. If the environment needs a secret to decode it's certificate, then it must provide the secret. If the secrets happen to be the same, then configuration would use the same UUID as the default (but we cannot assume that nor can we assume that the secret would be necessary). util: Add object checking for virObject{Ref|Unref} Rather than assuming that what's passed to virObject{Ref|Unref} would be a virObjectPtr as long as it's not NULL, let's do the similar checks virObjectIsClass in order to prevent a possible increment or decrement to some field at the obj->u.s.refs offset. 2017-08-15 John Ferlan util: Add magic number check for object validity The virObjectIsClass API has only ever checked object validity based on if the @obj is not NULL and it was derived from some class. While this has worked well in general, there is one additional check that could be made prior to calling virClassIsDerivedFrom which loops through the classes checking the magic number against the klass expected magic number. If by chance a non virObject is passed, rather than assuming the void * @obj is a _virObject and thus offsetting to obj->klass, obj->magic, and obj->parent, let's check that the void * @obj has at least the "base part" of the magic number in the right place and generate a more specific VIR_WARN message if not. There are many consumers to virObjectIsClass, include the locking primitives virObject{Lock|Unlock}, virObjectRWLock{Read|Write}, and virObjectRWUnlock. For those callers, the locking call will not fail, but it also will not attempt a virMutex* call which will "most likely" fail since the &obj->lock is used. In order to avoid some possible future wrap on the 0xCAFExxxx value, add a check during initialization that some new class won't cause the wrap. Should be good for a few years at least! It is still left up to the caller to handle the failed API calls just as it would be if it passed a NULL opaque pointer anyobj. 2017-08-15 John Ferlan util: Create common error path for invalid object If virObjectIsClass fails "internally" to virobject.c, create a macro to generate the VIR_WARN describing what the problem is. Also improve the checks and message a bit to indicate which was the failure - whether the obj was NULL or just not the right class util: Introduce and use virObjectRWUnlock Rather than overload virObjectUnlock as commit id '77f4593b' has done, create a separate virObjectRWUnlock API that will force the consumers to make the proper decision regarding unlocking the RWLock's. Similar to the RWLockRead and RWLockWrite, use the virObjectGetRWLockableObj helper. This restores the virObjectUnlock code to using the virObjectGetLockableObj. util: Introduce virObjectGetRWLockableObj Introduce a helper to handle the error path more cleanly. The same as virObjectGetLockableObj in order to essentially follow the original logic of commit 'b545f65d' to ensure that the input argument at least has some validity before using. 2017-08-15 John Ferlan util: Only have virObjectLock handle virObjectLockable Now that virObjectRWLockWrite exists to handle the virObjectRWLockable objects, let's restore virObjectLock to only handle virObjectLockable class locks. There still exists the possibility that the input @anyobj isn't a valid object and the resource isn't truly locked, but that also exists before commit id '77f4593b'. This also restores some logic that commit id '77f4593b' removed with respect to a common code path that commit id '10c2bb2b' had introduced as virObjectGetLockableObj. This code path merely does the same checks as the original virObjectLock commit 'b545f65d', but in callable/reusable helper to ensure the @obj at least has some validity before using. 2017-08-15 John Ferlan util: Introduce and use virObjectRWLockWrite Instead of making virObjectLock be the entry point for two different types of locks, let's create a virObjectRWLockWrite API which will only handle the virObjectRWLockableClass objects. Use the new virObjectRWLockWrite for the virdomainobjlist code in order to handle the Add, Remove, Rename, and Load operations that need to be very synchronous. 2017-08-15 John Ferlan util: Rename virObjectLockRead to virObjectRWLockRead Since the class it represents is based on virObjectRWLockableClass and in order to make sure we differentiate just in case anyone somehow believes they could use virObjectLockRead for a virObjectLockableClass, let's rename the API to use the RW in the name. Besides the RW locks refer to pthread_rwlock_{init|rdlock|wrlock|unlock|destroy} while the other locks refer to pthread_mutex_{init|lock|unlock|destroy}. 2017-08-14 Pavel Hrdina qemu: fix nwfilter deadlock in qemuProcessReconnect The correct lock order is: nwfilter driver lock (not used in this code path) nwfilter update lock virt driver lock (not used in this code path) domain object lock but the current code have this order: domain object lock nwfilter update lock 2017-08-14 Pavel Hrdina qemu: fix nwfilter deadlock while reverting to snapshot Introduced by commit <41127244fb90f08cf5032a5d7553f5f0390d925e>. conf: use virXMLPropString for Domain def parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for IOThread pin parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for IOThread parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for disk geometry parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: use virXMLPropString for KeyWrapCipherDef parsing XPath is good for random search of elements, not for accessing attributes of one node. conf: cleanup virDomainChrSourceDef parsing The @remaining variable is leftover from old code and it's not used anymore. use virXMLNodeNameEqual instead of xmlStrEqual util: introduce virXMLNodeNameEqual 2017-08-14 Martin Kletzander util: Rename virResctrl to virResctrlInfo This way later patches can add another structures with virResctrl prefix without the meaning being even more confusing than it needs to be. util: Make virResctrlGetCacheControlType() behave like other functions That means that returning negative values means error and non-negative values differ in meaning, but are all successful. Move resctrl-related code from conf/capabilities to util/virresctrl It doesn't access anything from conf/ and ti will be needed to use from other util/ places. This split makes the separation clearer. tests: Fix indentation in virfilewrapper.c virxml: Fix indentation 2017-08-14 Laine Stump util: eliminate superfluous saveVlan check in virNetDevSetNetConfig() Commit 81fb440b further qualified an if statement by adding the boolean saveVlan to the condition. Coverity pointed out that this change in the logic eliminated the need to check saveVlan in an argument to virAsprintf(). 2017-08-14 Laine Stump util: fix improper assignment of return value in virHostdevReadNetConfig() Commit 9a94af6d restructured virHostdevReadNetConfig() so that it would manually set ret = 0 after successfully reading the device's config, but Coverity pointed out that "ret = 0" was erroneously placed outside of an "else" clause, meaning that the the value of ret set in the "if" clause was unnecessarily and incorrectly overwritten. This patch moves ret = 0 into the else clause, which should silence Coverity. 2017-08-14 Guido Günther vbox: fix typo in warning message Acked-by: Laine Stump 2017-08-12 Laine Stump util: check for PF online status earlier in guest startup When using a VF from an SRIOV-capable network card in a guest (either in macvtap passthrough mode, or via VFIO PCI device assignment), The associated PF netdev must be online in order for the VF to be usable by the guest. The guest, however, is not able to change the state of the PF. And libvirt *could* set the PF online as needed, but that could lead to the host receiving unexpected IPv6 traffic (since the default for an unconfigured interface is to participate in IPv6 autoconf). For this reason, before assigning a VF to a guest, libvirt verifies that the related PF netdev is online - if it isn't, then we log an error and don't allow the guest startup to continue. Until now, this check was done during virNetDevSetNetConfig(). This works nicely because the same function is called both for macvtap passthrough and for VFIO device assignment. But in the case of VFIO, the VF has already been unbound from its netdev driver by the time we get to virNetDevSetNetConfig(), and in the case of dual port Mellanox NICs that have their VFs setup in single port mode, the *only* way to determine the proper PF netdev to query for online status is via the "phys_port_id" file that is in the VF netdev's sysfs directory. *BUT* if we've unbound the VF from the netdev driver, then it doesn't *have* a netdev sysfs directory. So, in order to check the correct PF netdev for online status, this patch moved the check earlier in the setup, into virNetDevSaveNetConfig(), which is called *before* unbinding the VF from its netdev driver. (Note that this implies that if you are using VFIO device assignment for the VFs of a Mellanox NIC that has the VFs programmed in single port mode, you must let the VFs be bound to their net driver and use "managed='yes'" in the device definition. To be more specific, this is only true if the VFs in single port mode are using port *2* of the PF - if the VFs are using only port 1, then the correct PF netdev will be arrived at by default/chance)) This resolves: https://bugzilla.redhat.com/267191 2017-08-12 Laine Stump util: restructure virNetDevReadNetConfig() to eliminate false error logs virHostdevRestoreNetConfig() calls virNetDevReadNetConfig() to try and read the "original config" of a netdev, and if that fails, it tries again with a different directory/netdev name. This achieves the desired effect (we end up finding the config wherever it may be), but for each failure, virNetDevReadNetConfig() places a nice error message in the system logs. Experience has shown that false-positive error logs like this lead to erroneous bug reports, and can often mislead those searching for *real* bugs. This patch changes virNetDevReadNetConfig() to explicitly check if the file exists before calling virFileReadAll(); if it doesn't exist, virNetDevReadNetConfig() returns a success, but leaves all the variables holding the results as NULL. (This makes sense if you define the purpose of the function as "read a netdev's config from its config file *if that file exists*). To take advantage of that change, the caller, virHostdevRestoreNetConfig() is modified to fail immediately if virNetDevReadNetConfig() returns an error, and otherwise to try the different directory/netdev name if adminMAC & vlan & MAC are all NULL after the preceding attempt. 2017-08-12 Laine Stump util: save the correct VF's info when using a dual port SRIOV NIC in single port mode Mellanox ConnectX-3 dual port SRIOV NICs present a bit of a challenge when assigning one of their VFs to a guest using VFIO device assignment. These NICs have only a single PCI PF device, and that single PF has two netdevs sharing the single PCI address - one for port 1 and one for port 2. When a VF is created it can also have 2 netdevs, or it can be setup in "single port" mode, where the VF has only a single netdev, and that netdev is connected either to port 1 or to port 2. When the VF is created in dual port mode, you get/set the MAC address/vlan tag for the port 1 VF by sending a netlink message to the PF's port1 netdev, and you get/set the MAC address/vlan tag for the port 2 VF by sending a netlink message to the PF's port 2 netdev. (Of course libvirt doesn't have any way to describe MAC/vlan info for 2 ports in a single hostdev interface, so that's a bit of a moot point) When the VF is created in single port mode, you can *set* the MAC/vlan info by sending a netlink message to *either* PF netdev - the driver is smart enough to understand that there's only a single netdev, and set the MAC/vlan for that netdev. When you want to *get* it, however, the driver is more accurate - it will return 00:00:00:00:00:00 for the MAC if you request it from the port 1 PF netdev when the VF was configured to be single port on port 2, or if you request if from the port 2 PF netdev when the VF was configured to be single port on port 1. Based on this information, when *getting* the MAC/vlan info (to save the original setting prior to assignment), we determine the correct PF netdev by matching phys_port_id between VF and PF. (IMPORTANT NOTE: this implies that to do PCI device assignment of the VFs on dual port Mellanox cards using (i.e. if you want the MAC address/vlan tag to be set), not only must the VFs be configured in single port mode, but also the VFs *must* be bound to the host VF net driver, and libvirt must use managed='yes') By the time libvirt is ready to set the new MAC/vlan tag, the VF has already been unbound from the host net driver and bound to vfio-pci. This isn't problematic though because, as stated earlier, when a VF is created in single port mode, commands to configure it can be sent to either the port 1 PF netdev or the port 2 PF netdev. When it is time to restore the original MAC/vlan tag, again the VF will *not* be bound to a host net driver, so it won't be possible to learn from sysfs whether to use the port 1 or port 2 PF netdev for the netlink commands. And again, it doesn't matter which netdev you use. However, we must keep in mind that we saved the original settings to a file called "${PF}_${VFNUM}". To solve this problem, we just check for the existence of ${PF1}_${VFNUM} and ${PF2}_${VFNUM}, and use whichever one we find (since we know that only one can be there) 2017-08-12 Laine Stump util: match phys_port_id when converting PF-netdev to/from VF-netdev This patch updates functions in netdev.c to pay attention to phys_port_id. It uses the new function virNetDevGetPhysPortID() to learn the phys_port_id of a VF or PF, then sends that info to virPCIGetNetName(), which has newly been modified to take an optional phys_port_id. 2017-08-12 Laine Stump util: make virPCIGetNetName() more versatile A single PCI device may have multiple netdevs associated with it. Each of those netdevs will have a different phys_port_id entry in sysfs. This patch modifies virPCIGetNetName() to allow selecting one of the potential many netdevs in two different ways: 1) by setting the "idx" argument, the caller can select the 1st (0), 2nd (1), etc. netdev from the PCI device's net subdirectory. 2) If the physPortID arg is set (to a null-terminated string) then virPCIGetNetName() returns the netdev that has that phys_port_id in the sysfs file of the same name in the netdev's directory. 2017-08-12 Laine Stump util: Fix const'ness of 1st arg to virPCIGetNetName() The first arg isn't modified in the function, so it should be const. 2017-08-12 Laine Stump util: new function virNetDevGetPhysPortID() On Linux each network device *can* (but not necessarily *does*) have an attribute called phys_port_id which can be read from the file of that name in the netdev's sysfs directory. The examples I've seen have been a many-digit hexadecimal number (as an ASCII string). This value can be useful when a single PCI device is associated with multiple netdevs (e.g a dual port Mellanox SR-IOV NIC - this card has a single PCI Physical Function (PF), and that PF has two netdevs associated with it (the "net" subdirectory of the PF in sysfs has two links rather than the usual single link to a netdev directory). Each of the PF netdevs has a different phys_port_id. The Virtual Functions (VF) are similar - the PF (a PCI device) has "n" VFs (also each of these is a PCI device), each VF has two netdevs, and each of the VF netdevs points back to the VF PCI device (with the "device" entry in its sysfs directory) as well as having a phys_port_id matching the PF netdev it is associated with. virNetDevGetPhysPortID() simply attempts to read the phys_port_id for the given netdev and return it to the caller. If this particular netdev driver doesn't support phys_port_id, it returns NULL (*not* a NULL-terminated string, but a NULL pointer) but still counts it as a success. 2017-08-11 intrigeri apparmor, libvirt-qemu: Allow QEMU to gather information about available host resources. 2017-08-11 Daniel P. Berrange m4: workaround clang/glibc problem with isnan() When building libvirt with clang we get bogus warnings about 'double' being promoted to 'long double' when calling isnan(). https://bugzilla.redhat.com/show_bug.cgi?id=1472437 Detect this broken isnan() / compiler combination and disable the -Wdouble-promotion flag. Reviewed-by: Pavel Hrdina 2017-08-10 Michal Privoznik qemuBuildMemoryBackendStr: Handle one more corner case https://bugzilla.redhat.com/show_bug.cgi?id=1458638 This code is so complicated because we allow enabling the same bits at many places. Just like in this case: huge pages can be enabled by global element under or on per basis. To complicate things a bit more, users are allowed to omit the page size which case the default page size is used. And this is what is causing this bug. If no page size is specified, @pagesize is keeping value of zero throughout whole function. Therefore we need yet another boolean to hold [use, don't use] information as we can't sue @pagesize for that. Reviewed-by: Martin Kletzander 2017-08-10 Michal Privoznik virhostdevtest: Don't leak @mgr->activeSCSIHostdevs So the hostdev manager has some lists to keep track which devices are active (=assigned to a domain) or inactive. The manager and its lists are allocated in myInit and freed in myCleanup but one of them (activeSCSIHostdevs) was missing. Also, the order in which the cleanup was done doesn't make it easy to spot it, therefore reoder it. vircgrouptest: Don't leak @cgroup In these test cases we create internal representation of cgroup, however, never free it. securityselinuxtest: Don't leak @mgr The security manager is created so that test cases can use it. However, it is never released. testCompareMemLock: Use correct free function for domain def virDomainDef is not an instance of virObject thus virObjectUnref() is not the correct function to be called. virDomainNetDefClear: Free @coalesce In virDomainNetDefParseXML() the def->coalesce is parsed and allocated by virDomainNetDefCoalesceParseXML() but in fact it's never freed . 2017-08-10 Michal Privoznik qemuDomainUndefineFlags: unlink nvram file regardless of domain state https://bugzilla.redhat.com/show_bug.cgi?id=1467245 Currently, there's a bug when undefining a domain with NVRAM store. Basically, the unlink() of the NVRAM store file happens during the undefine procedure iff domain is inactive. So, if domain is running and undefine is called the file is left behind. It won't be removed in the domain cleanup process either (qemuProcessStop). One of the solutions is to remove if regardless of the domain state and rely on qemu having the file opened. This still has a downside that if the domain is defined back the NVRAM store file is going to be new, any changes to the current one are lost (just like with any other file that is deleted while a process has it opened). But is it really a downside? 2017-08-08 Andrea Bolognani docs: Add "PCI topology and hotplug" guidelines For all machine types except i440fx, making a guest hotplug capable requires some sort of planning. Add some information to help users make educated choices when defining the PCI topology of guests. 2017-08-08 Ján Tomko introduce virConfReadString Rewrite virConfReadMem to take a null-terminated string. All the callers were calling strlen on it anyway. conf: check rombar against VIR_DOMAIN_TRISTATE_SWITCH_ABSENT Make the comparison explicit. 2017-08-08 Daniel P. Berrange docs: force content in , the XSTL generator will turn it into Device pci_0000_00_19_0 dettached Domain test started Device attached successfully error: Failed to start domain test2 error: Requested operation is not valid: PCI device 0000:00:19.0 is in use by domain test [ -- 1th time --] Device pci_0000_00_19_0 re-attached [ -- 2th time --] Device pci_0000_00_19_0 re-attached [ -- 3th time --] Device pci_0000_00_19_0 re-attached [ -- 4th time --] Device pci_0000_00_19_0 re-attached [ -- 5th time --] Device pci_0000_00_19_0 re-attached clean up Domain test destroyed Device pci_0000_00_19_0 re-attached The patch also fixes another problem, there won't be error like "qemuDomainReAttachHostdevDevices: Not reattaching active device 0000:00:19.0" in daemon log if some device is in active. As pciResetDevice and pciReattachDevice won't be called for the device anymore. This is sensible as we already reported error when preparing the device if it's active. Blindly trying to pciResetDevice & pciReattachDevice on the device and getting an error is just redundant. 2011-12-15 Osier Yang qemu: Honor the original properties of PCI device when detaching This patch fixes two problems: 1) The device will be reattached to host even if it's not managed, as there is a "pciDeviceSetManaged". 2) The device won't be reattached to host with original driver properly. As it doesn't honor the device original properties which are maintained by driver->activePciHostdevs. 2011-12-14 Wen Congyang spec: fix inverted logic on sanlock Commit d336dbdb tried to refactor sanlock to avoid building it on RHEL for architectures where it is not available, but used the wrong conditional. * libvirt.spec.in (with_sanlock): Use %ifarch, not %ifnarch. 2011-12-14 KAMEZAWA Hiroyuki virsh: support multifunction in attach-disk PCI can be specified by attach-disk but multifunction cannot be specified. Add --multifunction support. 2011-12-13 Eric Blake docs: tweak 'virsh edit' wording I was wondering why 'virsh edit' didn't support the same '--inactive' option as 'virsh dumpxml'; reading the source code showed that --inactive was already implied, and that the only way to alter a running guest rather than affecting next boot is by hot-plugging individual devices, or by something complex like saving the guest and modifying the save image. * tools/virsh.pod (define, edit): Mention behavior when guest is already running. 2011-12-13 Peter Krempa python: Fix export of virDomainSnapshotListChildrenNames Commit f2013c9dd1ce468b8620ee35c232a93ef7026fb0 added implementation of virDomainSnapshotListChildrenNames override export, but registration of the newly exported function was not added. *python/libvirt-override.c: - register export of function 2011-12-13 Lei Li Provide a helper method virDomainLiveConfigHelperMethod This chunk of code below repeated in several functions, factor it into a helper method virDomainLiveConfigHelperMethod to eliminate duplicated code based on Eric and Adam's suggestion. I have tested it for all the relevant APIs changed. 2011-12-13 Peter Krempa virsh: Print error message if argument parsing fails for cmdNodesuspend If parsing of arguments failed, virsh did silently exit returning and error state, but not specifying the possible problem. * tools/virsh: cmdNodesuspend: - error handling added 2011-12-13 Alex Jia tests: plug memory leak on linuxTestNodeInfo Detected by valgrind. Leak introduced in commit 82ff25e. * tests/nodeinfotest.c: avoid memory leak on nodeinfo test case. * how to reproduce? % cd tests && valgrind -v --leak-check=full ./nodeinfotest * actual valgrind result: ==22147== 65 bytes in 1 blocks are definitely lost in loss record 14 of 29 ==22147== at 0x4A0610F: realloc (vg_replace_malloc.c:525) ==22147== by 0x330D6FED94: __vasprintf_chk (in /lib64/libc-2.12.so) ==22147== by 0x426697: virVasprintf (stdio2.h:199) ==22147== by 0x426757: virAsprintf (util.c:1695) ==22147== by 0x41585F: linuxTestNodeInfo (nodeinfotest.c:108) ==22147== by 0x416B21: virtTestRun (testutils.c:141) ==22147== by 0x4157EA: mymain (nodeinfotest.c:140) ==22147== by 0x416217: virtTestMain (testutils.c:696) ==22147== by 0x330D61ECDC: (below main) (in /lib64/libc-2.12.so) ==22147== ==22147== LEAK SUMMARY: ==22147== definitely lost: 65 bytes in 1 blocks ==22147== indirectly lost: 0 bytes in 0 blocks ==22147== possibly lost: 0 bytes in 0 blocks ==22147== still reachable: 126,126 bytes in 1,341 blocks 2011-12-13 Osier Yang storage: Fix a potential crash when creating vol object If the vol object is newly created, it increases the volumes count, but doesn't decrease the volumes count when do cleanup. It can cause libvirtd to crash when one trying to free the volume objects like: for (i = 0; i < pool->volumes.count; i++) virStorageVolDefFree(pool->volumes.objs[i]); It's more reliable if we add the newly created vol object in the end. 2011-12-12 Eric Blake docs: document
elements in one place Improve the documentation of what forms a valid
element, since these elements appear in numerous devices. * docs/formatdomain.html.in (elementsAddress): New section. (elementsControllers, elementsUSB, elementsNICS, elementsInput) (elementsHub, elementsCharChannel, elementsSound): Refer to it. 2011-12-12 Eric Blake build: follow directory install conventions Commit 4d9e51f6 fixed a 'make uninstall' failure, but failed to follow other conventions already present in src/Makefile.am. In particular, we prefer MKDIR_P over mkdir -p, and should have a matching rmdir during uninstall for every directory created during install (the idea being that uninstall in a DESTDIR should be clean, while installation in the final system should not fail with non-empty directories left behind). * tools/Makefile.am (install-sysconfig, install-initscript) (install-systemd): Use MKDIR_P. (uninstall-sysconfig, uninstall-initscript, uninstall-systemd): Also remove directories. * daemon/Makefile.am (install-data-local, install-data-polkit) (install-logrotate, install-sysconfig, install-sysctl) (install-init-redhat, install-init-upstart, install-init-systemd) (install-data-sasl): Use MKDIR_P. (uninstall-data-polkit, uninstall-sysconfig, uninstall-sysctl) (uninstall-init-redhat, uninstall-init-upstart) (uninstall-init-systemd): Also remove directory. (uninstall-logrotate): New rule. (uninstall-local): Add uninstall-logrotate. 2011-12-12 Jiri Denemark qemu: Disable EOF processing during qemuDomainDestroy When destroying a domain qemuDomainDestroy kills its qemu process and starts a new job, which means it unlocks the domain object and locks it again after some time. Although the object is usually unlocked for a pretty short time, chances are another thread processing an EOF event on qemu monitor is able to lock the object first and does all the cleanup by itself. This leads to wrong shutoff reason and lifecycle event detail and virDomainDestroy API incorrectly reporting failure to destroy an inactive domain. Reported by Charlie Smurthwaite. 2011-12-12 Michal Privoznik virsh: Free returned MIME type string In terms of documentation to virDomainScreenshot, caller MUST free returned value. But virsh was not. 2011-12-12 Osier Yang Maint: Update AUTHORs Add Rommer in. 2011-12-12 Rommer storage: Activate/deactivate logical volumes only on local node Current "-ay | -an" has problems on pool starting/refreshing if the volumes are clustered. Rommer has posted a patch to list 2 months ago. https://www.redhat.com/archives/libvir-list/2011-October/msg01116.html But IMO we shouldn't skip the inactived vols. So this is a squashed patch by Rommer. 2011-12-12 Josh Durgin security: don't try to label network disks Network disks don't have paths to be resolved or files to be checked for ownership. ee3efc41e6233e625aa03003bf3127319ccd546f checked this for some image label functions, but was partially reverted in a refactor. This finishes adding the check to each security driver's set and restore label methods for images. 2011-12-12 Dave Allan Fix make uninstall Make uninstall currently fails with the following message: rmdir /etc/sasl2/ rmdir: failed to remove `/etc/sasl2/': Directory not empty That's fine (correct in fact) so force the command to return success with || : 2011-12-10 Laine Stump test: replace deprecated "fedora-13" machine with "pc-0.13" One of the xml tests in the test suite was created using a now-deprecated qemu machine type ("fedora-13", which was only ever valid for Fedora builds of qemu). Although strictly speaking it's not necessary to replace it with an actual supported qemu machine type (since the xml in question is never actually sent to qemu), this patch changes it to the actually-supported "pc-0.13" just for general tidiness. (Also, on some Fedora builds which contain a special patch to rid the world of "fedora-13", having it mentioned in the test suite will cause make check to fail.) 2011-12-10 Laine Stump network: don't add iptables rules for externally managed networks This patch addresses https://bugzilla.redhat.com/show_bug.cgi?id=760442 When a network has any forward type other than route, nat or none, the network configuration should be done completely external to libvirt - libvirt only uses these types to allow configuring guests in a manner that isn't tied to a specific host (all the host-specific information, in particular interface names, port profile data, and bandwidth configuration is in the network definition, and the guest configuration only references it). Due to a bug in the bridge network driver, libvirt was adding iptables rules for networks with forward type='bridge' etc. any time libvirtd was restarted while one of these networks was active. This patch eliminates that error by only "reloading" iptables rules if forward type is route, nat, or none. 2011-12-09 Michael Ellerman qemu: Prepare to cater for more general address assignment Currently qemuDomainAssignPCIAddresses() is called to assign addresses to PCI devices. We need to do something similar for devices with spapr-vio addresses. So create one place where address assignment will be done, that is qemuDomainAssignAddresses(). 2011-12-09 Michael Ellerman qemu: Add address in qemuBuildChrDeviceStr() on pseries For the PPC64 pseries machine type we need to add address information for the spapr-vty device. qemu: Use spapr-vscsi on pseries machine type On the PPC64 pseries machine type we need to use the spapr-vscsi device rather than an lsi. 2011-12-09 Eric Blake network: allow '-' in model name In QEMU PPC64 we have a network device called "spapr-vlan". We can specify this using the existing syntax for network devices, however libvirt currently rejects "spapr-vlan" in virDomainNetDefParseXML() because of the "-". Fix the code to accept "-". * src/conf/domain_conf.c (virDomainNetDefParseXML): Allow '-' in model name, and be more efficient. * docs/schemas/domaincommon.rng: Limit valid model names to match code. Based on a patch by Michael Ellerman. 2011-12-09 Michal Privoznik threadpool: Use while loop on virCondWait instead of simple 'if' statement as virCondWait can return even if associated condition was not signaled. threads: Document spurious wakeups on virCondWait 2011-12-09 Alex Jia virsh: plug memory leak on cmdDomblklist Detected by valgrind. Leak introduced in commit 88a993b: * tools/virsh.c: fix memory leak on cmdDomblklist. * how to reproduce? % valgrind -v --leak-check=full virsh domblklist * actual valgrind result: ==6573== 1,836 bytes in 1 blocks are definitely lost in loss record 110 of 124 ==6573== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==6573== by 0x330D71497D: xdr_string (in /lib64/libc-2.12.so) ==6573== by 0x4D26CED: xdr_remote_nonnull_string (remote_protocol.c:30) ==6573== by 0x4D28138: xdr_remote_domain_get_xml_desc_ret (remote_protocol.c:1418) ==6573== by 0x4D3C0C2: virNetMessageDecodePayload (virnetmessage.c:382) ==6573== by 0x4D3279F: virNetClientProgramCall (virnetclientprogram.c:382) ==6573== by 0x4D0D50B: callWithFD (remote_driver.c:4339) ==6573== by 0x4D0D5AB: call (remote_driver.c:4360) ==6573== by 0x4D16EAF: remoteDomainGetXMLDesc (remote_client_bodies.h:861) ==6573== by 0x4CF9F4F: virDomainGetXMLDesc (libvirt.c:4098) ==6573== by 0x4154D9: cmdDomblklist (virsh.c:1722) ==6573== by 0x4149E2: vshCommandRun (virsh.c:16365) ==6573== ==6573== 46,009 (352 direct, 45,657 indirect) bytes in 1 blocks are definitely lost in loss record 123 of 124 ==6573== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==6573== by 0x3318286DC6: xmlXPathNewContext (in /usr/lib64/libxml2.so.2.7.6) ==6573== by 0x4C79AE2: virXMLParseHelper (xml.c:779) ==6573== by 0x415512: cmdDomblklist (virsh.c:1726) ==6573== by 0x4149E2: vshCommandRun (virsh.c:16365) ==6573== by 0x427743: main (virsh.c:17867) ==6573== ==6573== LEAK SUMMARY: ==6573== definitely lost: 2,188 bytes in 2 blocks ==6573== indirectly lost: 45,657 bytes in 332 blocks ==6573== possibly lost: 0 bytes in 0 blocks ==6573== still reachable: 128,034 bytes in 1,364 blocks ==6573== suppressed: 0 bytes in 0 blocks 2011-12-09 Stefan Berger fix error when parsing ppc64 models on x86 host When parsing ppc64 models on an x86 host an out-of-memory error message is displayed due to it checking for retcpus being NULL. Fix this by removing the check whether retcpus is NULL since we will realloc into this variable. Also in the X86 model parser display the OOM error at the location where it happens. 2011-12-09 Stefan Berger fix memory leak in src/nodeinfo.c Fix memory leak: ==27534== 24 bytes in 1 blocks are definitely lost in loss record 207 of 530 ==27534== at 0x4A05E46: malloc (vg_replace_malloc.c:195) ==27534== by 0x38EC26EC37: vasprintf (in /lib64/libc-2.13.so) ==27534== by 0x4E998E6: virVasprintf (util.c:1677) ==27534== by 0x4E999F1: virAsprintf (util.c:1695) ==27534== by 0x4F1EAAC: nodeGetInfo (nodeinfo.c:593) ==27534== by 0x47948F: qemuCapsInitCPU (qemu_capabilities.c:855) ==27534== by 0x4796B1: qemuCapsInit (qemu_capabilities.c:915) ==27534== by 0x456550: qemuCreateCapabilities (qemu_driver.c:245) ==27534== by 0x4578C4: qemudStartup (qemu_driver.c:580) ==27534== by 0x4F20886: virStateInitialize (libvirt.c:852) ==27534== by 0x420E55: daemonRunStateInit (libvirtd.c:1156) ==27534== by 0x4E94C56: virThreadHelper (threads-pthread.c:157) Mark this leaked variable as const char * when it is passed into another function. 2011-12-09 Michal Privoznik threadpool: Don't wait on condition if pool has no workers Pool creates new workers dynamically. However, it is possible for a pool to have no workers. If we want to free that pool, we don't want to wait on quit condition as it will never be signaled. 2011-12-09 Jiri Denemark bridge: Fix forward delay APIs Due to copy&paste error in c1df2c14b590b3d68b707aa4f3a570f95a6bc548, virNetDevBridge[SG]etSTPDelay APIs were accessing wrong file. 2011-12-09 Peter Krempa cpu: Add cpu flags supported by newest qemu Add support for newly supported Intel cpu features. Newly supported flags are: pclmuldq, dtes64, smx, fma, pdcm, movbe, xsave, osxsave and avx. This adds support for Intel's Sandy Bridge platform. virsh: return correct value from cmdDomIfGetLink Reported by Alex Jia . Function cmdDomIfGetLink did not set a success return value on success path. 2011-12-09 Stefan Berger Pass the VM's UUID into the nwfilter subsystem A preparatory patch for DHCP snooping where we want to be able to differentiate between a VM's interface using the tuple of . We assume that MAC addresses could possibly be re-used between different networks (VLANs) thus do not only want to rely on the MAC address to identify an interface. At the current 'final destination' in virNWFilterInstantiate I am leaving the vmuuid parameter as ATTRIBUTE_UNUSED until the DHCP snooping patches arrive. (we may not post the DHCP snooping patches for 0.9.9, though) Mostly this is a pretty trivial patch. On the lowest layers, in lxc_driver and uml_conf, I am passing the virDomainDefPtr around until I am passing only the VM's uuid into the NWFilter calls. 2011-12-09 Stefan Berger nwfilter: cleanup return codes in nwfilter subsystem This patch cleans up return codes in the nwfilter subsystem. Some functions in nwfilter_conf.c (validators and formatters) are keeping their bool return for now and I am converting their return code to true/false. All other functions now have failure return codes of -1 and success of 0. [I searched for all occurences of ' 1;' and checked all 'if ' and adapted where needed. After that I did a grep for 'NWFilter' in the source tree.] 2011-12-09 Alex Jia virsh: plug memory leak on cmdDomIfGetLink() sucessful path Detected by valgrind. Leak introduced in commit dc675f3: * tools/virsh.c: fix memory leak on cmdDomIfGetLink. * how to reproduce? % valgrind -v --leak-check=full virsh domif-getlink 0 * actual valgrind result: ==13102== 18 bytes in 1 blocks are definitely lost in loss record 9 of 47 ==13102== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==13102== by 0x322A6A67DD: xmlStrndup (in /usr/lib64/libxml2.so.2.7.6) ==13102== by 0x414892: cmdDomIfGetLink (virsh.c:1538) ==13102== by 0x4136A2: vshCommandRun (virsh.c:16363) ==13102== by 0x4253FB: main (virsh.c:17865) ==13102== ==13102== LEAK SUMMARY: ==13102== definitely lost: 18 bytes in 1 blocks ==13102== indirectly lost: 0 bytes in 0 blocks ==13102== possibly lost: 0 bytes in 0 blocks ==13102== still reachable: 127,888 bytes in 1,361 blocks ==13102== suppressed: 0 bytes in 0 blocks 2011-12-09 Alex Jia virsh: plug memory leak on cmdBlkdeviotune() sucessful path Detected by valgrind. Leak introduced in commit e9bd9a0: * tools/virsh.c: fix memory leak on cmdBlkdeviotune. * how to reproduce? % valgrind -v --leak-check=full virsh blkdeviotune * actual valgrind result: ==12759== 576 bytes in 1 blocks are definitely lost in loss record 18 of 29 ==12759== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==12759== by 0x42134E: _vshCalloc.clone.2 (virsh.c:422) ==12759== by 0x4217CB: cmdBlkdeviotune (virsh.c:6364) ==12759== by 0x4136A2: vshCommandRun (virsh.c:16363) ==12759== by 0x4253FB: main (virsh.c:17865) ==12759== ==12759== LEAK SUMMARY: ==12759== definitely lost: 576 bytes in 1 blocks ==12759== indirectly lost: 0 bytes in 0 blocks ==12759== possibly lost: 0 bytes in 0 blocks ==12759== still reachable: 126,964 bytes in 1,342 blocks ==12759== suppressed: 0 bytes in 0 blocks 2011-12-08 Eric Blake maint: allow bootstrap in a sandbox Jiri Denemark reported an instance of bootstrapping libvirt failing when run inside a sandbox, traced to rpm trying to access /var/ which was not permitted by the sandbox. Alex Jia reported that 0.9.8-rc1 failed to bootstrap if patch(1) is not installed. * bootstrap.conf (buildreq): Avoid rpm call if python-config exists. Also, require patch, in case we have gnulib-local diffs. 2011-12-08 Laine Stump test: fix potential lock corruption in test driver In some error situations, the function testDomainRestoreFlags() could unlock the test driver mutex without first locking it. This patch moves the lock operation earlier, so that it occurs before any potential jump down to the unlock call. I found this problem while auditing the test driver lock usage to determine the cause of a hang while running the following test: cd tests; while true; do printf x; ./undefine; done This patch *does not* solve that problem, but we now understand its actual source, and danpb is working on a patch. 2011-12-08 Eric Blake spec: fix logic bug in deciding to turn on cgconfig https://bugzilla.redhat.com/show_bug.cgi?id=738725 Commit ecd8725 tried to silence a spurious warning on the initial libvirt install, and commit ba6cbb1 tried to fix up the logic to the correct Fedora version, but the warning was still present due to a logic bug: since %{fedora} and %{rhel} are never simulatanously set, then 0%{rhel} <= 6 made the %if always true. Checking for minimum versions (via >=) is okay, but checking for maximum versions (via <=) requires a prerequisite test that the platform being tested is non-zero. Also fix a bogus setting of with_libxl (although we previously hard-code with_libxl to 0 for rhel earlier in the file, so this was not as severe a bug). * libvirt.spec.in (with_cgconfig): Don't enable cgconfig on F16. 2011-12-08 Eric Blake spec: make it easier to autoreconf when building rpm Over time, Fedora and RHEL RPMs have often backported upstream patches that touched configure.ac and/or Makefile.am; this necessitates rerunning the autotools for the patch to be effective. Making this a one-liner spec tweak will make it easier for future backports to pull patches without having to find all the places to touch to properly use the autotools. Meanwhile, there have been historical instances where an update in the autotools caused FTBFS situations, so this is not on by default. * libvirt.spec.in (enable_autotools): New variable, default off. (BuildRequires): Conditionally add autotools. (%build): Conditionally use them before configure. * mingw32-libvirt.spec.in: Likewise. 2011-12-08 Daniel P. Berrange When checking nttyFDs to see if it is != 1, be sure to use '1' and not '-1' * src/lxc/lxc_controller.c: Fix check for tty count 2011-12-08 Daniel P. Berrange Fix installation of libvirt-guests.service The installation rules for the libvirt-guests.service were totally broken - Installing in the wrong location - The location was not overridable - The install-systemd rule was not invoked anywhere - The install-systemd rule was not invoking install-initscript which it depends on - The installed service file lacked a .service extension * tools/Makefile.am: Fix install of libvirt-guests.service 2011-12-08 Daniel P. Berrange Ensure to prefix %{buildroot} when overriding systemd install location The %makeinstall macro does not set DESTDIR, instead of explicitly prefixes %{buildroot} onto all paths. Thus we need to do the same when setting the systemd unit dir * libvirt.spec.in: Prefix %{buildroot} onto %{unitdir} 2011-12-08 Bharata B Rao Add ppc64 specific definitions to domain.rng ppc64 as new arch type and pseries as new machine type are added under ... . 2011-12-08 Prerna Saxena Clean up qemuBuildCommandLine to remove x86-specific assumptions from generic code. This implements the minimal set of changes needed in libvirt to launch a PowerPC-KVM based guest. It removes x86-specific assumptions about choice of serial driver backend from generic qemu guest commandline generation code. It also restricts the ACPI capability to be available for an x86 or x86_64 domain. This is not a complete solution -- it still does not guarantee libvirt the capability to flag non-supported options in guest XML. (Eg, an ACPI specification in a PowerPC guest XML will still get processed, even though qemu-system-ppc64 does not support it while qemu-system-x86_64 does.) This drawback exists because libvirt falls back on qemu to query supported features, and qemu '-h' blindly lists all capabilities -- irrespective of whether they are available while emulating a given architecture or not. The long-term solution would be for qemu to list out capabilities based on architecture and platform -- so that libvirt can cleanly make out what devices are supported on an arch (say 'ppc64') and platform (say, 'mac99'). 2011-12-08 Prerna Saxena Add support for ppc64 qemu This enables libvirt to select the correct qemu binary (qemu-system-ppc64) for a guest vm based on arch 'ppc64'. Also, libvirt is enabled to correctly parse the list of supported PowerPC CPUs, generated by running 'qemu-system-ppc64 -cpu ?' Acked-by: Stefan Berger 2011-12-08 Prerna Saxena Modify the tests/nodeinfotest.c to use sysfs in addition to proc/cpuinfo This patch creates a new sysfs hierarchy under tests/nodeinfodata/linux-nodeinfo-sysfs-test-1. Output files and /proc/cpuinfo files are also respectively added for both x86 and ppc64. 2011-12-08 Prerna Saxena Use sysfs to gather host topology, in place of /proc/cpuinfo Libvirt at present depends on /proc/cpuinfo to gather host details such as CPUs, cores, threads, etc. This is an architecture- dependent approach. An alternative is to use 'Sysfs', which provides a platform-agnostic interface to parse host CPU topology. 2011-12-08 Christophe Fergeau maint: move my name to commiters Since I have commit rights on libvirt-glib, I can also push to libvirt, Eric Blake told to move my name up to committers to better reflect reality. 2011-12-08 Daniel Veillard Release of libvirt-0.9.8 * configure.ac docs/news.html.in libvirt.spec.in: updated for the release * po/*.po*: fetched localization update and regenerated 2011-12-08 Eric Blake spec: don't use chkconfig --list https://bugzilla.redhat.com/show_bug.cgi?id=694403 reports that the specfile is incorrectly checking for a running libvirt-guests service. For example, $ LC_ALL=es_ES chkconfig --list libvirt-guests libvirt-guests 0:desactivado 1:desactivado 2:desactivado 3:activo 4:activo 5:activo 6:desactivado will fail to find 5:on, even though it is active. But chkconfig already has a mode where you can silently use the exit status to check for an active service. * libvirt.spec.in (%post): Use simpler chkconfig options, to avoid issues with localization. 2011-12-08 Eric Blake build: fix build with older libxml2 On RHEL 5, with libxml2-2.6.26, the build failed with: virsh.c: In function 'vshNodeIsSuperset': virsh.c:11951: warning: implicit declaration of function 'xmlChildElementCount' (or if warnings aren't errors, a link failure later on). * src/util/xml.h (virXMLChildElementCount): New prototype. * src/util/xml.c (virXMLChildElementCount): New function. * src/libvirt_private.syms (xml.h): Export it. * tools/virsh.c (vshNodeIsSuperset): Use it. 2011-12-08 Daniel P. Berrange Fix updating of haveTheBuck in RPC client to be race-free When one thread passes the buck to another thread, it uses virCondSignal to wake up the target thread. The variable 'haveTheBuck' is not updated in a race-free manner when this occurs. The current thread sets it to false, and the woken up thread sets it to true. There is a window where a 3rd thread can come in and grab the buck. Even if this didn't lead to crashes & deadlocks, this would still result in unfairness in the buckpassing algorithm. A better solution is to *never* set haveTheBuck to false when we're passing the buck. Only set it to false when there is no further thread waiting for the buck. * src/rpc/virnetclient.c: Only set haveTheBuck to false if no thread is waiting 2011-12-08 Daniel P. Berrange Revert fd066925440ba48acc95d8f31b2c98b1cc9d582d Commit fd066925440ba48acc95d8f31b2c98b1cc9d582d tried to fix a race condition in commit fa9595003d043df9f2efe95521c00898cef27106 Author: Daniel P. Berrange Date: Fri Nov 11 15:28:41 2011 +0000 Explicitly track whether the buck is held in remote client Unfortunately there is a second race condition whereby the event loop can trigger due to incoming data to read. Revert this fix, so a complete fix for the problem can be cleanly applied * src/rpc/virnetclient.c: Revert fd066925440ba48acc95d8f31b2c98b1cc9d582d 2011-12-07 Jim Fehlig Prevent crash of libvirtd when attaching to existing qemu process With security_driver set to "none" in /etc/libvirt/qemu.conf, libvirtd would crash when attempted to attach to an existing qemu process. Only copy the security model if it actually exists. 2011-12-07 Christophe Fergeau Add documentation for Fix typo in virDomainResume API doc It's referring to virSuspendDomain instead of virDomainSuspend. 2011-12-07 Jiri Denemark qemu: Ignore shutdown event from destroyed domain During virDomainDestroy, QEMU may emit SHUTDOWN event as a response to SIGTERM and since domain object is still locked, the event is processed after the domain is destroyed. We need to ignore this event in such case to avoid changing domain state from shutoff to shutdown. 2011-12-07 Osier Yang npiv: Expose fabric_name outside This patch is to expose the fabric_name of fc_host class, which might be useful for users who wants to known which fabric the (v)HBA connects to. The patch also adds the missed capabilities' XML schema of scsi_host, (of course, with fabric_wwn added), and update the documents (docs/formatnode.html.in) 2011-12-07 Daniel P. Berrange Conditionalize daemonPath decl for Win32 which lacks UNIX sockets 2011-12-07 Daniel P. Berrange Improve error reporting when libvirtd is not installed Currently if you try to connect to a local libvirtd when libvirtd is not in $PATH, you'll get an error error: internal error invalid use of command API This is because remoteFindDaemonPath() returns NULL, which causes us to pass NULL into virNetSocketConnectUNIX which in turn causes us to pass NULL into virCommandNewArgList. Adding missing error checks improves this to error: internal error Unable to locate libvirtd daemon in $PATH * src/remote/remote_driver.c: Report error if libvirtd cannot be found * src/rpc/virnetsocket.c: Report error if caller requested spawning of daemon, but provided no binary path 2011-12-05 Eric Blake spec: fix sanlock dependency * libvirt.spec.in (with_sanlock): On RHEL, don't force sanlock on architectures where it isn't available. 2011-12-05 Eric Blake spec: add dmidecode as prereq https://bugzilla.redhat.com/show_bug.cgi?id=754909 complains that because libvirt didn't require dmidecode, that the logs are noisy and virConnectGetSysinfo needlessly fails. Even 'virt-what' requires dmidecode, so it's not that onerous of a dependency. We may be able to drop this in the future when we move to parsing sysfs data, but for now, listing the dependency will help matters. * libvirt.spec.in (Requires): Sort Requires before BuildRequires. Add dmidecode. 2011-12-05 Eric Blake build: reduce warnings from older gcc Older gcc warns (on every file!) that -Wabi and -Wdeprecated only make sense on C++ projects. Newer gcc accepts these warnings for C, but it is not clear that they can do anything useful, so it is easier to just drop the warnings altogether. * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Silence -Wabi and -Wdeprecated on older gcc. Reported by Peter Krempa. 2011-12-05 Jiri Denemark examples: Update event tests for shutdown event 2011-12-05 Daniel P. Berrange Fix incorrect symbols for virtime.h module breaking Mingw32 The Mingw32 linker highlighted that the symbols for virtime.h declared in libvirt_private.syms were incorrect * src/libvirt_private.syms: Fix virtime.h symbols 2011-12-05 Jiri Denemark qemu: Rework handling of shutdown event When QEMU guest finishes its shutdown sequence, qemu stops virtual CPUs and when started with -no-shutdown waits for us to kill it using SGITERM. Since QEMU is flushing its internal buffers, some time may pass before QEMU actually dies. We mistakenly used "paused" state (and events) for this which is quite confusing since users may see a domain going to pause while they expect it to shutdown. Since we already have "shutdown" state with "the domain is being shut down" semantics, we should use it for this state. However, the state didn't have a corresponding event so I created one and called its detail as VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED (guest OS finished its shutdown sequence) with the intent to add VIR_DOMAIN_EVENT_SHUTDOWN_STARTED in the future if we have a sufficiently capable guest agent that can notify us when guest OS starts to shutdown. 2011-12-05 Guido Günther remote_driver: don't fail if keepalive check fails Otherwise connections to older libvirt abort with: $ virsh -c qemu+ssh://host.example.com/system list error: invalid connection pointer in virDrvSupportsFeature error: failed to connect to the hypervisor Tested against 0.8.3 and 0.9.8-rc2. 2011-12-05 Jiri Denemark Add support for QEMU 1.0 2011-12-05 Peter Krempa python: Expose binding for virNodeGetMemoryStats() This patch adds binding for virNodeGetMemoryStats method of libvirtd. Return value is represented as a python dictionary mapping field names to values. python: Expose binding for virNodeGetCPUStats() This patch adds binding for virNodeGetCPUStats method of libvirtd. Return value is represented as a python dictionary mapping field names to values. 2011-12-04 Eric Blake maint: fix improper use of 'an' https://bugzilla.redhat.com/show_bug.cgi?id=648855 mentioned a misuse of 'an' where 'a' is proper; that has since been fixed, but a search found other problems (some were a spelling error for 'and', while most were fixed by 'a'). * daemon/stream.c: Fix grammar. * src/conf/domain_conf.c: Likewise. * src/conf/domain_event.c: Likewise. * src/esx/esx_driver.c: Likewise. * src/esx/esx_vi.c: Likewise. * src/rpc/virnetclient.c: Likewise. * src/rpc/virnetserverprogram.c: Likewise. * src/storage/storage_backend_fs.c: Likewise. * src/util/conf.c: Likewise. * src/util/dnsmasq.c: Likewise. * src/util/iptables.c: Likewise. * src/xen/xen_hypervisor.c: Likewise. * src/xen/xend_internal.c: Likewise. * src/xen/xs_internal.c: Likewise. * tools/virsh.c: Likewise. 2011-12-03 Eric Blake build: require more tools from maintainers We want our tarballs to be complete - this means that any generated file that gets shipped as part of the tarball so that ordinary users don't have to rebuild it must be something that the maintainer can generate. There have been various reports of random build failures when using libvirt.git instead of a tarball, and often it is due to missing a maintainer-specific tool to produce one of these generated files. This patch raises the bar for what you must have installed to build libvirt.git, but does not impact what you can get away with for building tarballs. Note: It still remains possible to do a successful 'make dist' without these tools, when starting from a release tarball. * bootstrap.conf (buildreq): Add tools that maintainers need for a successful 'make dist' from a fresh git checkout. 2011-12-03 Eric Blake command: handle empty buffer argument correctly virBufferContentAndReset (intentionally) returns NULL for a buffer with no content, but it is feasible to invoke a command with an explicit empty string. * src/util/command.c (virCommandAddEnvBuffer): Reject empty string. (virCommandAddArgBuffer): Allow explicit empty argument. * tests/commandtest.c (test9): Test it. * tests/commanddata/test9.log: Adjust. 2011-12-03 Eric Blake build: fix build on Cygwin The RPC fixups needed on Linux are also needed on cygwin, and worked without further tweaking to the list of fixups. Also, unlike BSD, Cygwin exports 'struct ifreq', but unlike Linux, Cygwin lacks the ioctls that we were using 'struct ifreq' to access. This patch allows compilation under cygwin. * src/rpc/genprotocol.pl: Also perform fixups on cygwin. * src/util/virnetdev.c (HAVE_STRUCT_IFREQ): Also require AF_PACKET definition. * src/util/virnetdevbridge.c (virNetDevSetupControlFull): Only compile if SIOCBRADDBR works. 2011-12-02 Eric Blake build: fix build at -O2 on rawhide I had previously tested commit 059d746 with -O intentionally omitted from my CFLAGS; but that means that I missed out on this warning from gcc 4.6.2 when optimizations are enabled: util/buf.c: In function 'virBufferGetIndent': util/buf.c:86:1: error: function might be candidate for attribute 'pure' [-Werror=suggest-attribute=pure] While it is probably a good idea to add the attributes and silence this warning, it's also invasive; 'make -k' found more than 75 such complaints. And it doesn't help that gcc 4.6.2 is still buggy (coreutils reported a case where gcc 4.6.2 incorrectly suggested marking a function pure that incremented a global variable; fixed in gcc 4.7). So the best fix for now is to disable the warning. It also doesn't help that I stumbled across another problem - gcc documents that -Wsuggest-attribute=pure only warns if you use -O, or if you use -fipa-pure-const. But in practice, when I omitted -O but added -fipa-pure-const, the warnings are fickle - I got warnings for simple compilation that disappeared when I also added -fPIC. And the way libtool compiles things is with -fPIC first, then without -fPIC but with errors sent to /dev/null - which meant that without disabling -Wsuggest-attribute=pure, I got a compile error with no message. :( See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10197 * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Silence -Wsuggest-attribute warnings for now. 2011-12-02 Serge Hallyn apparmor: allow tunnelled migrations. The pathname for the pipe for tunnelled migration is unresolvable. The libvirt apparmor driver therefore refuses access, causing migration to fail. If we can't resolve the path, the worst that can happen is that we should have given permission to the file but didn't. Otherwise (especially since this is a /proc/$$/fd/N file) the file is already open and libvirt won't be refused access by apparmor anyway. Also adjust virt-aa-helper to allow access to the *.tunnelmigrate.dest.name files. For more information, see https://launchpad.net/bugs/869553. 2011-12-02 Peter Krempa client: Check if other thread claims it has the buck before claiming it. Originaly, the code checked if another client is the queue and infered ownership of the buck from that. Commit fa9595003d043df9f2efe95521c008 added a separate variable to track the buck. That caused, that a new call might enter claiming it has the buck, while another thread was signalled to take the buck. This ends in two threads claiming they hold the buck and entering poll(). This happens due to a race on waking up threads on the client lock mutex. This caused multi-threaded clients to hang, most prominently visible and reproducible on python based clients, like virt-manager. This patch causes threads, that have been signalled to take the buck to re-check if buck is held by another thread. 2011-12-02 Stefan Berger Update of filters to handle multiple IP addresses With fragments borrowed from David Steven's previous submission and some further modifications: A set of modifications to filters to handle multiple IP addresses (and MAC addresses) per interface. Also: - enable DHCP traffic from VM to any DHCP server - will require an update to a libvirt-tck data file 2011-12-02 Eric Blake virsh: translate net-info help Reported by kato.tomoyuki@jp.fujitsu.com at https://bugzilla.redhat.com/show_bug.cgi?id=749564 * tools/virsh.c (info_network_info): Mark string for translation. 2011-12-02 Eric Blake maint: typo fixes Many of these were mentioned by Yuri Chornoivan in: https://bugzilla.redhat.com/show_bug.cgi?id=669506 * src/esx/esx_vi.c (esxVI_WaitForTaskCompletion): Fix spelling. * src/conf/netdev_vport_profile_conf.c (virNetDevVPortProfileParse): Likewise. * src/xen/xend_internal.c (xenDaemonDomainSetVcpusFlags): Likewise. * src/xen/xm_internal.c (xenXMDomainSetVcpusFlags): Likewise. * src/esx/esx_util.c (esxUtil_ResolveHostname): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendFileSystemBuild): Likewise. * daemon/libvirtd.conf: Likewise. * src/util/logging.c (virLogMessage): Likewise. * src/uml/uml_conf.c (umlBuildCommandLineNet): Likewise. * src/vmx/vmx.c (virVMXFormatEthernet): Likewise. 2011-12-01 Eric Blake build: update to latest gnulib * .gnulib: Update to latest, for improved 'make syntax-check' and compiler warnings. * m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Re-silence -Wformat-nonliteral. * cfg.mk (_test_script_regex): Recognize our test scripts. * gnulib/local/lib/*.diff: Drop, now that gnulib has this. * tests/virsh-optparse: Fix use of compare. * tests/virsh-schedinfo: Likewise. 2011-12-01 Eric Blake build: properly skip tests I got this failure on FreeBSD: shunloadtest.c: In function 'main': shunloadtest.c:150: error: 'EXIT_AM_SKIP' undeclared (first use in this function) but inspection showed several other problems, all fixed here. * tests/domainsnapshotxml2xmltest.c [!WITH_QEMU]: Ensure EXIT_AM_SKIP is defined. * tests/esxutilstest.c [!WITH_ESX]: Likewise. * tests/openvzutilstest.c [!WITH_OPENVZ]: Likewise. * tests/qemuargv2xmltest.c [!WITH_QEMU]: Likewise. * tests/qemuhelptest.c [!WITH_QEMU]: Likewise. * tests/qemuxml2argvtest.c [!WITH_QEMU]: Likewise. * tests/qemuxml2xmltest.c [!WITH_QEMU]: Likewise. * tests/qemuxmlnstest.c [!WITH_QEMU]: Likewise. * tests/shunloadtest.c [!linux]: Likewise. * tests/vmx2xmltest.c [!WITH_VMX]: Likewise. * tests/xml2vmxtest.c [!WITH_VMX]: Likewise. 2011-12-01 Daniel P. Berrange Avoid crash in shunloadtest For unknown reasons, the shunloadtest will crash on Fedora 16 inside dlopen() (gdb) bt #0 0x00000000000050e6 in ?? () #1 0x00007ff61a77b9d5 in floor () from /lib64/libm.so.6 #2 0x00007ff61e522963 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ff61e5297e6 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ff61e52917a in _dl_open () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ff61e0f6f26 in dlopen_doit () from /lib64/libdl.so.2 #7 0x00007ff61e525006 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ff61e0f752f in _dlerror_run () from /lib64/libdl.so.2 #9 0x00007ff61e0f6fc1 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2 #10 0x0000000000400a15 in main (argc=, argv=) at shunloadtest.c:105 Changing from RTLD_NOW to RTLD_LAZY avoids this problem, but quite possibly does not fix the root cause. * shunloadtest.c: s/NOW/LAZY/ 2011-12-01 Daniel P. Berrange Fix build for platforms lacking struct ifreq This ought to fix the build if you have net/if.h but do not have struct ifreq * configure.ac: Check for struct ifreq in net/if.h * src/util/virnetdev.c: Conditionalize to avoid use of struct ifreq if it does not exist 2011-12-01 Eric Blake build: fix 'make dist' without dtrace probes.h can only be generated on Linux, and then only with dtrace installed. If it is part of the tarball, then either 'make dist' will fail if you don't have that setup, or we would have to start keeping probes.h in libvirt.git. Since we only need it to be generated when dtrace is in use, it's better to avoid shipping it in the first place, and avoid tracking it in git. Meanwhile, there is a build dependency - since the RPC code is generated, it can be built early; but when dtrace is enabled, we must ensure probes.h is built even earlier. Commit 1afcfbdd tried to fix this, but did so in a way that added probes.h into the tarball, and broke VPATH as well. Commit ecbca767 fixed VPATH, but didn't fix the more fundamental problem. This patch solves the issue by adding a dependency instead. Tested with 'make dist' in a clean VPATH builds, for both './configure --without-dtrace' and './configure --with-dtrace'; all configurations were able to correctly build a tarball, and the dtrace configuration no longer sticks probes.h in the tarball. * src/Makefile.am (REMOTE_DRIVER_GENERATED): Don't ship probes.h; rather, make it a dependency. 2011-12-01 Lei Li Fix a logic error for setting block I/O Fix a logic error, the initial value of ret = -1, if just set --config, it will goto endjob directly without doing its really job here. 2011-12-01 Daniel P. Berrange Don't use undocumented __isleap macro The glibc time.h header has an undocumented __isleap macro that we are using. Since it is undocumented & does not appear on any other OS, stop using it and just define the macro in libvirt code instead. * src/util/virtime.c: Remove __isleap usage 2011-12-01 Michal Privoznik virsh: Allow other escape characters for console Currently virsh supports only ^] as escape character for console. However, some users might want to use something else. This patch creates such ability by specifying '-e' switch on virsh command line. 2011-12-01 Peter Krempa bridge_driver: Don't define network if XML contains more IPv4 adreses. Only one IPv4 DHCP definition is supported. Originally the code checked for a multiple definition and returned an error, but the new domain definition was already added to networks. This patch moves the check before the newly defined network is added to active networks. *src/network/bridge_driver.c: networkDefine(): - move multiple IPv4 addresses check before definition is used. 2011-12-01 Eric Blake spec: mark directories in /var/run as ghosts We have several directories that are created on the fly, and which only contain state relevant to a running libvirtd process (all located in /var/run). Since the directories are created as needed, and make no sense without a running libvirtd, we want them deleted if libvirt is uninstalled. And in F15 and newer, /var/run is on tmpfs (forcing us to recreate on the fly); which means that someone trying to verify a complete rpm will fail if the directory does not currently exist because libvirtd has not been started since boot. The solution, then, is to mark the directories as %ghost, so that rpm knows that we own them and will clean it up if libvirt is uninstalled, but will no longer create the directory for us at install, nor complain at verify time if the directory does not exist. See https://bugzilla.redhat.com/show_bug.cgi?id=656611. * libvirt.spec.in (%files): Add %ghost to temporary directories that we don't install, but want cleaned up on libvirt removal. 2011-12-01 Jiri Denemark virsh: Fix possible deadlock when virsh is about to exit Not only was ctl->quit accessed without a mutex but unfortunately, virEventAddTimeout only interrupts the poll when event loop is running so the hack needs to add a timeout that will make next poll return immediately without blocking. 2011-12-01 Alex Jia util: Plug memory leak on virNetDevMacVLanCreateWithVPortProfile() error path Detected by Coverity. Leak introduced in commit 90074ec. 2011-12-01 Alex Jia util: Plug memory leak on virNetDevBridgeGet() sucessful path Detected by Coverity. Leak introduced in commit c1df2c1. Two bugs here: 1. memory leak on successful parse 2. failure to parse still returned success 2011-12-01 Alex Jia uml: Plug memory leak on umlStartVMDaemon() error path Detected by Coverity. Leak introduced in commit 8866eed. Two bugs here: 1. logfd wasn't closed on all return paths 2. if we failed to mark a domain autodestroy, then the domain was not made transient but we still returned success 2011-12-01 Alex Jia rpc: Plug memory leak on virNetClientSendInternal() error path Detected by Coverity. Leak introduced in commit 673adba. Two separate bugs here: 1. call was not freed on all error paths 2. virCondDestroy was called even if virCondInit failed 2011-11-30 Alex Jia conf: Plug memory leak on virDomainDefParseXML() error path Detected by Coverity. Leak introduced in commit 0873b68. qemu: Plug memory leak onqemuProcessWaitForMonitor() error path Detected by Coverity. Leak introduced in commit 109efd7. 2011-11-30 Prerna Saxena Add PPC cpu driver. To add support for running libvirt on PowerPC, a CPU driver for the PowerPC platform must be added. Most generic cpu driver routines such as CPU compare, decode, etc are based on CPUID comparison and are not relevant for non-x86 platforms. Here, we introduce stubs for relevant PowerPC routines invoked by libvirt. 2011-11-30 Hu Tao qemu: filter blkio 0-device-weight at two other places filter 0-device-weight when: - getting blkio parameters with --config - starting up a domain When testing with blkio, I found these issues: (dom is down) virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,500 virsh blkiotune dom --device-weights /dev/sda,300,/dev/sdb,0 virsh blkiotune dom weight : 800 device_weight : /dev/sda,200,/dev/sdb,0 # issue 1: shows 0 device weight of /dev/sdb that may confuse user (continued) virsh start dom # issue 2: If /dev/sdb doesn't exist, libvirt refuses to bring the # dom up because it wants to set the device weight to 0 of a # non-existing device. Since 0 means no weight-limit, we really don't # have to set it. 2011-11-30 Eric Blake qemu: amend existing table of device weights Prior to this patch, for a running dom, the commands: $ virsh blkiotune dom --device-weights /dev/sda,502,/dev/sdb,498 $ virsh blkiotune dom --device-weights /dev/sda,503 $ virsh blkiotune dom weight : 500 device_weight : /dev/sda,503 claim that /dev/sdb no longer has a non-default weight, but directly querying cgroups says otherwise: $ cat /cgroup/blkio/libvirt/qemu/dom/blkio.weight_device 8:0 503 8:16 498 After this patch, an explicit 0 is required to remove a device path from the XML, and omitting a device path that was previously specified leaves that device path untouched in the XML, to match cgroups behavior. * src/qemu/qemu_driver.c (parseBlkioWeightDeviceStr): Rename... (qemuDomainParseDeviceWeightStr): ...and use correct type. (qemuDomainSetBlkioParameters): After parsing string, modify rather than replacing existing table. * tools/virsh.pod (blkiotune): Tweak wording. 2011-11-30 Eric Blake conf: reject duplicate paths in device weights The next patch will make it possible to have virDomainSetBlkioParameters leave device weights unchanged if they are not mentioned in the incoming string, but this only works if the list of block weights does not allow duplicate paths. Technically, a user can still confuse libvirt by passing alternate spellings that resolve to the same device, but it is not worth worrying about working around that kind of abuse. * src/conf/domain_conf.c (virDomainDefParseXML): Require unique paths. 2011-11-30 Hu Tao virsh: fix setting weight and device-weights at the same time When setting both blkio weight and device-weights at the same time, the weight is lost. Fix it. 2011-11-30 Lei Li Add tests for blkdeviotune Support virDomain{Set, Get}BlockIoTune in the python API Python support for both setting and getting block I/O throttle. Enable the blkdeviotune command in virsh Support virsh command blkdeviotune. Can set or query a block disk I/O throttle setting. Implement virDomain{Set, Get}BlockIoTune for the qemu driver Implement the block I/O throttle setting and getting support to qemu driver. Support block I/O throttle in XML Enable block I/O throttle for per-disk in XML, as the first per-disk IO tuning parameter. Add virDomain{Set, Get}BlockIoTune support to the remote driver Support Block I/O Throttle setting and query to remote driver. 2011-11-30 Daniel P. Berrange Fix leak build config file path * src/libvirt.c: Free user directory path 2011-11-30 Daniel P. Berrange Remove time APIs from src/util/util.h The virTimestamp and virTimeMs functions in src/util/util.h duplicate functionality from virtime.h, in a non-async signal safe manner. Remove them, and convert all code over to the new APIs. * src/util/util.c, src/util/util.h: Delete virTimeMs and virTimestamp * src/lxc/lxc_driver.c, src/qemu/qemu_domain.c, src/qemu/qemu_driver.c, src/qemu/qemu_migration.c, src/qemu/qemu_process.c, src/util/event_poll.c: Convert to use virtime APIs 2011-11-30 Daniel P. Berrange Make logging async signal safe wrt time stamp generation Use the new virTimeStringNowRaw() API for generating log timestamps in an async signal safe manner * src/util/logging.c: Use virTimeStringNowRaw 2011-11-30 Daniel P. Berrange Add internal APIs for dealing with time The logging APIs need to be able to generate formatted timestamps using only async signal safe functions. This rules out using gmtime/localtime/malloc/gettimeday(!) and much more. Introduce a new internal API which is async signal safe. virTimeMillisNowRaw replacement for gettimeofday. Uses clock_gettime where available, otherwise falls back to the unsafe gettimeofday virTimeFieldsNowRaw replacements for gmtime(), convert a timestamp virTimeFieldsThenRaw into a broken out set of fields. No localtime() replacement is provided, because converting to local time is not practical with only async signal safe APIs. virTimeStringNowRaw replacements for strftime() which print a timestamp virTimeStringThenRaw into a string, using a pre-determined format, with a fixed size buffer (VIR_TIME_STRING_BUFLEN) For each of these there is also a version without the Raw postfix which raises a full libvirt error. These versions are not async signal safe * src/Makefile.am, src/util/virtime.c, src/util/virtime.h: New files * src/libvirt_private.syms: New APis * configure.ac: Check for clock_gettime in -lrt * tests/virtimetest.c, tests/Makefile.am: Test new APIs 2011-11-30 Daniel P. Berrange Remove obsolete virGetPMCapabilities sym from private symbols file Fix the build on Mingw32 by removing the now obsolete virGetPMCapabilities symbol from the private exports file * src/libvirt_private.syms: Remove virGetPMCapabilities 2011-11-30 Daniel P. Berrange Don't mark suspend as active until we know it is running If suspend failed for some reason (e.g. too short duration) then subsequent attempts to trigger suspend were rejected because we had already marked a suspend as being in progress * src/util/virnodesuspend.c: Don't mark suspend as active until we've successfully triggered it 2011-11-30 Daniel P. Berrange Add suspend info to Xen, LXC and UML hypervisor capabilities * src/lxc/lxc_conf.c, src/uml/uml_conf.c, src/xen/xen_hypervisor.c: Initialize suspend capabilities * tests/xencapsdata/*xml: Add empty powermgmt capabilities 2011-11-30 Daniel P. Berrange Remove pointless strdup in node suspend code The command name for the suspend action does not need to be strdup'd. The constant string can be used directly. This also means the code can be trivially rearranged to make the switch clearer * src/util/virnodesuspend.c: Remove strdup of cmdString 2011-11-30 Daniel P. Berrange Do lazy init of host PM features To avoid probing the host power management features on any call to virInitialize, only initialize the mutex in virNodeSuspendInit. Do lazy load of the supported PM target mask when it is actually needed * src/util/virnodesuspend.c: Lazy init of supported features 2011-11-30 Daniel P. Berrange Remove powerMgmt_valid field from capabilities struct If we ensure that virNodeSuspendGetTargetMask always resets *bitmask to zero upon failure, there is no need for the powerMgmt_valid field. * src/util/virnodesuspend.c: Ensure *bitmask is zero upon failure * src/conf/capabilities.c, src/conf/capabilities.h: Remove powerMgmt_valid field * src/qemu/qemu_capabilities.c: Remove powerMgmt_valid 2011-11-30 Daniel P. Berrange Add export of node suspend capabilities APIs * src/libvirt_private.syms: Export virNodeSuspendSupportsTarget and virNodeSuspendGetTargetMask 2011-11-30 Daniel P. Berrange Move suspend capabilities APIs out of util.h into virnodesuspend.c The node suspend capabilities APIs should not have been put into util.[ch]. Instead move them into virnodesuspend.[ch] * src/util/util.c, src/util/util.h: Remove suspend capabilities APIs * src/util/virnodesuspend.c, src/util/virnodesuspend.h: Add suspend capabilities APIs * src/qemu/qemu_capabilities.c: Include virnodesuspend.h 2011-11-30 Daniel P. Berrange Rename suspend capabilities APIs Rename virGetPMCapabilities to virNodeSuspendGetTargetMask and virDiscoverHostPMFeature to virNodeSuspendSupportsTarget. * src/util/util.c, src/util/util.h: Rename APIs * src/qemu/qemu_capabilities.c, src/util/virnodesuspend.c: Adjust for new names 2011-11-30 Daniel P. Berrange Sanitize virDiscoverHostPMFeature to return a boolean Since virDiscoverHostPMFeature is just checking one feature, there is no reason for it to return a bitmask. Change it to return a boolean * src/util/util.c, src/util/util.h: Make virDiscoverHostPMFeature return a boolean 2011-11-30 Daniel P. Berrange Move the virHostPMCapability enum helpers into capabilities.c The virHostPMCapability enum helper was declared in util.h but implemented in capabilities.c, which is in a completely separate library at link time. Move the declaration into the capabilities.c file and rename it to match normal conventions * src/util/util.h: Remove virHostPMCapability enum decl * src/conf/capabilities.c: Add virCapsHostPMTarget enum 2011-11-30 Daniel P. Berrange Fix capabilities XML to use generic terms for suspend targets The capabilities XML uses the x86 specific terms 'S3', 'S4' and 'Hybrid-Syspend'. Switch it to use the same terminology as the API constants and virsh options, eg 'suspend_mem' 'suspend_disk' and 'suspend_hybrid' * docs/formatcaps.html.in, docs/schemas/capability.rng, src/conf/capabilities.c: Rename suspend constants 2011-11-30 Daniel P. Berrange Remove internal only virHostPMCapability enum The internal virHostPMCapability enum just duplicates the public virNodeSuspendTarget enum, but with different names. * src/util/util.c: Use VIR_NODE_SUSPEND_TARGET constants * src/util/util.h: Remove virHostPMCapability enum * src/conf/capabilities.c: Use VIR_NODE_SUSPEND_TARGET_LAST 2011-11-30 Daniel P. Berrange Fix values of PM target type constants The VIR_NODE_SUSPEND_TARGET constants are not flags, so they should just be assigned straightforward incrementing values. * include/libvirt/libvirt.h.in: Change VIR_NODE_SUSPEND_TARGET values * src/util/virnodesuspend.c: Fix suspend target checks 2011-11-30 Alex Jia util: avoid null deref on qcowXGetBackingStore Detected by Coverity. the only case is caller passes a NULL to 'format' variable, then taking 'if (format)' false branch, the function qcow2GetBackingStoreFormat will directly dereferences the NULL 'format' pointer variable. 2011-11-30 Alex Jia virsh: correct return value error Fix cmdDomblklist to return 'true' on success instead of '0' https://bugzilla.redhat.com/show_bug.cgi?id=758590 2011-11-30 Lei Li Add new API virDomain{Set, Get}BlockIoTune This patch add new pulic API virDomainSetBlockIoTune and virDomainGetBlockIoTune. 2011-11-29 Hu Tao blkiotune: add qemu support for blkiotune.device_weight Implement setting/getting per-device blkio weights in qemu, using the cgroups blkio.weight_device tunable. 2011-11-29 Hu Tao blkiotune: add interface for blkiotune.device_weight This adds per-device weights to . Note that the cgroups implementation only supports weights per block device, and not per-file within the device; hence this option must be global to the domain definition rather than tied to individual / entries: /path/to/block 1000 .. This patch also adds a parameter --device-weights to virsh command blkiotune for setting/getting blkiotune.weight_device for any hypervisor that supports it. All entries under are concatenated into a single string attribute under virDomain{Get,Set}BlkioParameters, named "device_weight". 2011-11-29 Eric Blake qemu: fix blkiotune --live --config Without this, 'virsh blkiotune --live --config --weight=n' only affected live. * src/qemu/qemu_driver.c (qemuDomainSetBlkioParameters): Allow setting both configurations at once. 2011-11-29 Eric Blake qemu, lxc: drop redundant checks After the previous patch, there are now some redundant checks. * src/qemu/qemu_driver.c (qemudDomainGetVcpuPinInfo) (qemuGetSchedulerParametersFlags): Drop checks now guaranteed by libvirt.c. * src/lxc/lxc_driver.c (lxcGetSchedulerParametersFlags): Likewise. 2011-11-29 Eric Blake API: prevent query of --live and --config at once Drivers were inconsistent when presented both --live and --config at once. For example, within qemu, getting memory parameters favored live, getting blkio tuning favored config, and getting scheduler parameters errored out. Also, some, but not all, attempts to mix flags on query were filtered at the virsh level. We shouldn't have to duplicate efforts in every client app, nor in every driver. So, it is simpler to just enforce that the two flags cannot both be used at once on query operations, which has precedent in libvirt.c, and which matches the documentation of virDomainModificationImpact. * src/libvirt.c (virDomainGetMemoryParameters) (virDomainGetBlkioParameters) (virDomainGetSchedulerParametersFlags, virDomainGetVcpuPinInfo): Borrow sanity checking from virDomainGetVcpusFlags. 2011-11-29 Eric Blake build: fix typo in last patch * src/remote_protocol-structs: Use correct RPC name. 2011-11-29 Osier Yang block_resize: Update test file for RPC Pushed under build failure rule. block_resize: Expose the new API to virsh 2011-11-29 Osier Yang block_resize: Implement qemu driver method It requires the domain is running, otherwise fails. Resize to a lower size is supported, but should be used with extreme caution. In order to prohibit the "size" overflowing after multiplied by 1024. We do checking in the codes. For QMP mode, the default units is Bytes, the passed size needs to be multiplied by 1024, however, for HMP mode, the default units is "Megabytes", the passed "size" needs to be divided by 1024 then. 2011-11-29 Osier Yang block_resize: Implement qemu monitor functions Implements functions for both HMP and QMP mode. For HMP mode, qemu uses "M" as the units by default, so the passed "sized" is divided by 1024. For QMP mode, qemu uses "Bytes" as the units by default, the passed "sized" is multiplied by 1024. All of the monitor functions return -1 on failure, 0 on success, or -2 if not supported. 2011-11-29 Osier Yang block_resize: Wire up the remote protocol 2011-11-29 Osier Yang block_resize: Define the new API The new API is named as "virDomainBlockResize", intending to add support for qemu monitor command "block_resize" (both HMP and QMP). Similar with APIs like "virDomainSetMemoryFlags", the units for argument "size" is kilobytes. 2011-11-29 Michal Privoznik bandwidth: Fix funky identation 2011-11-29 Jiri Denemark virsh: Fix typos in man page lxc: Fix suspend/resume with freezer cgroup 2011-11-29 Osier Yang examples: Correct the example command to use testnode.xml * s/-connection/-c/ * Removes the redundant '/'. * Add "absolute" so that it's more clear. Pushed under trivial rule. 2011-11-29 Srivatsa S. Bhat Add virsh command to initiate suspend on the host Add a new command 'nodesuspend' to perform a timed suspend on the host. 2011-11-29 Srivatsa S. Bhat Implement the core API to suspend/resume the host Add the core functions that implement the functionality of the API. Suspend is done by using an asynchronous mechanism so that we can return the status to the caller before the host gets suspended. This asynchronous operation is achieved by suspending the host in a separate thread of execution. However, returning the status to the caller is only best-effort, but not guaranteed. To resume the host, an RTC alarm is set up (based on how long we want to suspend) before suspending the host. When this alarm fires, the host gets woken up. Suspend-to-RAM operation on a host running Linux can take upto more than 20 seconds, depending on the load of the system. (Freezing of tasks, an operation preceding any suspend operation, is given up after a 20 second timeout). And Suspend-to-Disk can take even more time, considering the time required for compaction, creating the memory image and writing it to disk etc. So, we do not allow the user to specify a suspend duration of less than 60 seconds, to be on the safer side, since we don't want to prematurely declare failure when we only had to wait for some more time. 2011-11-29 Srivatsa S. Bhat Add the remote protocol implementation for virNodeSuspendForDuration Implement the remote protocol for virNodeSuspendForDuration() API. Add a public API to invoke suspend/resume on the host Implement the public definitions for the new API virNodeSuspendForDuration() which will be subsequently used to do a timed suspend on the host. Add 'Hybrid-Suspend' power management discovery for the host Some systems support a feature known as 'Hybrid-Suspend', apart from the usual system-wide sleep states such as Suspend-to-RAM (S3) or Suspend-to-Disk (S4). Add the functionality to discover this power management feature and export it in the capabilities XML under the tag. 2011-11-29 Jiri Denemark rpc: Really send non-blocking calls while waiting for another call When another thread was dispatching while we wanted to send a non-blocking call, we correctly queued the call and woke up the thread but the thread just threw the call away since it forgot to recheck if its socket was writable. 2011-11-28 Michal Privoznik virsh: Don't traverse childless nodes in vshNodeIsSuperset If both nodes do not have any children, we pass zero to virBitmapAlloc which returns NULL. In turn we report OOM error and return false (meaning nodes are different). This is not true. 2011-11-28 Christian Franke virnetsocket: pass XAUTORITY for ssh connection When spawning an ssh connection, the environment variables DISPLAY, SSH_ASKPASS, ... are passed. However XAUTHORITY, which is necessary if the .Xauthority is in a non default place, was not passed. 2011-11-28 Lorin Hochstein conf: make virt-xml-validate work with vbox domains virt-xml-validate fails when run on a domain XML file of type 'vbox'. For failing test case, see https://bugzilla.redhat.com/show_bug.cgi?id=757097 This patch updates the XML schema to accept all valid hypervisor types, as well as dropping hypervisor types that are not in use by the current code base. 2011-11-28 Michal Privoznik conf: Improve incorrect root element error messages When user pass wrong root element, it is not 'internal error' and we can give him hint what we are expecting. 2011-11-28 Daniel P. Berrange Refactor initial LXC mem tune / device ACL code To make lxcSetContainerResources smaller, pull the mem tune and device ACL setup code out into separate methods * src/lxc/lxc_controller.c: Introduce lxcSetContainerMemTune and lxcSetContainerDeviceACL 2011-11-28 Daniel P. Berrange Add support for blkio tuning of LXC containers * src/lxc/lxc_controller.c: Refactor setting of initial blkio tuning parameters * src/lxc/lxc_driver.c: Enable live change of blkio tuning Add support for CPU quota/period to LXC driver * src/lxc/lxc_driver.c: Support changing quota/period for LXC containers * src/lxc/lxc_controller.c: Set initial quota/period at startup 2011-11-28 Daniel P. Berrange Support CPU placement in LXC driver While LXC does not have the concept of VCPUS, so we can't do per-VCPU pCPU placement, we can support the VM level CPU placement. Todo this simply set the CPU affinity of the LXC controller at startup. All child processes will inherit this affinity. * src/lxc/lxc_controller.c: Set process affinity 2011-11-28 Daniel P. Berrange Support NUMA memory placement for LXC containers Use numactl to set NUMA memory placement for LXC containers * src/lxc/lxc_controller.c: Support NUMA memory placement 2011-11-25 Michal Privoznik storage: Refetch file status after open This partly reverts my previous patch f88de3eb. We need to get file status after open, as given path could have been symlink, so fstat() will operate on different file than lstat(). 2011-11-25 Paolo Bonzini util: fix thinko in runIO When aligning you need to clear the bits in the mask and leave the others aside. Likely this code has never run, and will never run. 2011-11-25 Michal Privoznik storage: Skip socket and fifo on pool-start If pool directory contains special files like FIFO or sockets we want to skip those on pool-start or pool-refresh otherwise open() will get an error. conf: Don't drop console definition on domain restart One of my latest patches 2e37bf42d28d8bb5d045b206587c64643c64d02a copy serial console definition. On domain shutdown we save this info into state XML. However, later on the daemon start we simply drop this info and since we are not re-reading qemu log, vm->def->consoles[0] does not get populated with copy. Therefore we need to avoid dropping console definition if it is just alias for serial console. 2011-11-24 Jiri Denemark Fix version numbers for isAlive and setKeepAlive driver APIs build: Properly generate and check virkeepaliveprotocol-structs This fixes make dist broken by recent keepalive series examples: Use virConnectOpenAuth in events-c qemu: Cancel p2p migration when connection breaks If a connection to destination host is lost during peer-to-peer migration (because keepalive protocol timed out), we won't be able to finish the migration and it doesn't make sense to wait for qemu to transmit all data. This patch automatically cancels such migration without waiting for virDomainAbortJob to be called. qemu: Add support for keepalive messages during p2p migration Add keepalive support into domain-events examples Implement virConnectIsAlive in all drivers Introduce virConnectIsAlive API This API can be used to check if the socket associated with virConnectPtr is still open or it was closed (probably because keepalive protocol timed out). If there the connection is local (i.e., no socket is associated with the connection, it is trivially always alive. Implement keepalive protocol in remote driver Add support for async close of client RPC socket Implement keepalive protocol in libvirt daemon virsh: Always run event loop Since virsh already implements event loop, it has to also run it. So far the event loop was only running during virsh console command. Introduce virConnectSetKeepAlive virConnectSetKeepAlive public API can be used by a client connecting to remote server to start using keepalive protocol. The API is handled directly by remote driver and not transmitted over the wire to the server. Implement common keepalive handling These APIs are used by both client and server RPC layer to handle processing of keepalive messages. Define keepalive protocol The keepalive program has two procedures: PING, and PONG. Both are used only in asynchronous messages and the sender doesn't wait for any reply. However, the party which receives PING messages is supposed to react by sending PONG message the other party, but no explicit binding between PING and PONG messages is made. For backward compatibility neither server nor client are allowed to send keepalive messages before checking that remote party supports them. rpc: Add some debug messages to virNetClient rpc: Fix handling of non-blocking calls that could not be sent When virNetClientIOEventLoop is called for a non-blocking call and not even a single byte can be sent from this call without blocking, we properly reported that to the caller which properly frees the call. But we never removed the call from a call queue. rpc: Fix a typo in virNetClientSendNonBlock documentation rpc: Pass the buck only to the first available thread 2011-11-23 Stefan Berger nwfilter: remove virConnectPtr from internal API calls Remove the virConnectPtr from the nwfilter's internal API calls as far as possible. 2011-11-23 Peter Krempa qemu: Avoid dereference of NULL pointer If something fails while initializing qemu job object in qemuDomainObjPrivateAlloc(), memory to the private pointer is freed, but after that, the pointer is still dereferenced, which may result in a segfault. * qemuDomainObjPrivateAlloc() - Don't dereference NULL pointer. 2011-11-23 Eric Blake qemu: fix a const-correctness issue Generally, functions which return malloc'd strings should be typed as 'char *', not 'const char *', to make it obvious that the caller is responsible to free things. free(const char *) fails to compile, and although we have a cast embedded in VIR_FREE to work around poor code that frees const char *, it's better to not rely on that hack. * src/qemu/qemu_driver.c (qemuDiskPathToAlias): Change return type. (qemuDomainBlockJobImpl): Update caller. 2011-11-23 Eric Blake API: prefer 'disk' over 'block' or 'path' Given that we can now handle the target's disk shorthand, in addition to an absolute path to the file or block device used on the host, the term 'disk' fits a bit better as the parameter name than 'path'. * include/libvirt/libvirt.h.in: Update some parameter names. * src/libvirt.c (virDomainBlockStats, virDomainBlockStatsFlags) (virDomainBlockPeek, virDomainGetBlockInfo, virDomainBlockJobAbort) (virDomainGetBlockJobInfo, virDomainBlockJobSetSpeed) (virDomainBlockPull): Likewise. 2011-11-23 Eric Blake blockstats: support lookup by path in blockstats Commit 89b6284f made it possible to pass either a source name or the target device to most API demanding a disk designation, but forgot to update the documentation. It also failed to update virDomainBlockStats to take both forms. This patch fixes both the documentation and the remaining function. Xen continues to use just device shorthand (that is, I did not implement path lookup there, since xen does not track a domain_conf to quickly tie a path back to the device shorthand). * src/libvirt.c (virDomainBlockStats, virDomainBlockStatsFlags) (virDomainGetBlockInfo, virDomainBlockPeek) (virDomainBlockJobAbort, virDomainGetBlockJobInfo) (virDomainBlockJobSetSpeed, virDomainBlockPull): Document acceptable disk naming conventions. * src/qemu/qemu_driver.c (qemuDomainBlockStats) (qemuDomainBlockStatsFlags): Allow lookup by source name. * src/test/test_driver.c (testDomainBlockStats): Likewise. 2011-11-23 Michal Privoznik nwfilter: Initialize virNWFilterAddIpAddrForIfname return variable Latest nwfilter patch ad6c67cf introduced uninitialized return value. This was spotted by 4.6.2 gcc. 2011-11-23 Daniel P. Berrange Fix disabling of virtual port profile code on old hosts The WITH_VIRTUALPORT macro is defined to 0 when disabled, not left undefined. So #if must be used instead of #ifdef * src/util/virnetdevvportprofile.c: s/#ifdef/#if/ 2011-11-23 Daniel P. Berrange Fix uninitialized variable in NWfilter IP learning code * src/nwfilter/nwfilter_learnipaddr.c: Initialize ret variable 2011-11-23 Michal Prívozník virsh: Increase device-detach intelligence Up to now users have to give a full XML description on input when device-detaching. If they omitted something it lead to unclear error messages (like generated MAC wasn't found, etc.). With this patch users can specify only those information which specify one device sufficiently precise. Remaining information is completed from domain. 2011-11-23 Stefan Berger Enable detection of multiple IP addresses In preparation of DHCP Snooping and the detection of multiple IP addresses per interface: The hash table that is used to collect the detected IP address of an interface can so far only handle one IP address per interface. With this patch we extend this to allow it to handle a list of IP addresses. Above changes the returned variable type of virNWFilterGetIpAddrForIfname() from char * to virNWFilterVarValuePtr; adapt all existing functions calling this function. 2011-11-22 Eli Qiao fix error message when using wrong URI alias When configuring a URI alias like this in 'libvirt.conf': uri_aliases = [ "jj#j=qemu+ssh://root@127.0.0.1/system", "sleet=qemu+ssh://root@sleet.cloud.example.com/system", ] virsh -c jj#j It will show this error message: 'no connection driver available for No connection for URI jj#j' Actually,we expect this message below: Malformed 'uri_aliases' config entry 'jj#j=qemu+ssh://root@127.0.0.1/system', aliases may only contain 'a-Z, 0-9, _, -' Give this patch to fix this error. 2011-11-22 Stefan Berger Pass additional parameter into applyDHCPOnly function In preparation for the DHCP Snooping code: Pass an additional parameter into the applyDHCPOnly function of the 'techdriver'. 2011-11-22 Stefan Berger nwfilter: use shell variable to invoke 'ip(6)tables' command Introduce a shell variable 'IBT' to invoke the ip(6)tables command. Tested with libvirt-tck. 2011-11-22 Stefan Berger nwfilter: use shell variable to invoke 'ebtables' command Introduce a shell variable 'EBT' to invoke the ebtables command. Hard-code the used ebtables table to '-t nat'. Tested with libvirt-tck. 2011-11-22 Stefan Berger Improve error reporting of failures to apply filtering rules Display the executed command and failure message if a command failed to execute. Add documentation for STP filtering support Add documentation for the STP filtering support. Describe the XML attributes that are supported. Add test cases for STP traffic filtering This patch adds a few test cases for the XML parsing of STP filtering nodes. Add support for STP filtering This patch adds support for filtering of STP (spanning tree protocol) traffic to the parser and makes us of the ebtables support for STP filtering. This code now enables the filtering of traffic in chains with prefix 'stp'. Add a 'mac' chain With hunks borrowed from one of David Steven's previous patches, we now add the capability of having a 'mac' chain which is useful to filter for multiple valid MAC addresses. 2011-11-22 Eric Blake docs: fix grammar of capabilities * docs/formatcaps.html.in: Avoid run-on sentence, wrap lines. 2011-11-22 Daniel P. Berrange Add strings.h include to capabilities.h for ffs() function prototype On Mingw32 the ffs() function was not declared due to missing header include * src/conf/capabilities.c: The ffs() function lives in strings.h 2011-11-22 Osier Yang build: Update AUTHORS Add Chang Liu to the AUTHORS' list. 2011-11-22 Chang Liu storage: Fallback to use lvchange first if lvremove fails virStorageBackendLogicalDeleteVol() could not remove the lv with error "could not remove open logical volume" sometimes. Generally it's caused by the volume is still active, even if lvremove tries to remove it with option "--force". This patch is to fix it by disbale the lv first using "lvchange -aln" and "lvremove -f" afterwards if the direct "lvremove -f" failed. 2011-11-22 Srivatsa S. Bhat Export KVM Host Power Management capabilities This patch exports KVM Host Power Management capabilities as XML so that higher-level systems management software can make use of these features available in the host. The script "pm-is-supported" (from pm-utils package) is run to discover if Suspend-to-RAM (S3) or Suspend-to-Disk (S4) is supported by the host. If either of them are supported, then a new tag "" is introduced in the XML under the tag. However in case the query to check for power management features succeeded, but the host does not support any such feature, then the XML will contain an empty tag. In the event that the PM query itself failed, the XML will not contain any "power_management" tag. To use this, new APIs could be implemented in libvirt to exploit power management features such as S3/S4. 2011-11-21 Eric Blake conf: don't modify cpu set string during parsing None of the callers cared if str was updated to point to the next byte after the parsed cpuset; simplifying this results in quite a few code simplifications. Additionally, virCPUDefParseXML was strdup()'ing a malloc()'d string; avoiding a memory copy resulted in less code. * src/conf/domain_conf.h (virDomainCpuSetParse): Alter signature. * src/conf/domain_conf.c (virDomainCpuSetParse): Don't modify str. (virDomainVcpuPinDefParseXML, virDomainDefParseXML): Adjust callers. * src/conf/cpu_conf.c (virCPUDefParseXML): Likewise. * src/xen/xend_internal.c (sexpr_to_xend_topology): Likewise. * src/xen/xm_internal.c (xenXMDomainPinVcpu): Likewise. * src/xenxs/xen_sxpr.c (xenParseSxpr): Likewise. * src/xenxs/xen_xm.c (xenParseXM): Likewise. 2011-11-21 Roopa Prabhu qemu: don't release network actual device twice For direct attach devices, in qemuBuildCommandLine, we seem to be freeing actual device on error path (with networkReleaseActualDevice). But the actual device is not deleted. qemuProcessStop eventually deletes the direct attach device and releases actual device. But by the time qemuProcessStop is called qemuBuildCommandLine has already freed actual device, leaving stray macvtap devices behind on error. So the simplest fix is to remove the networkReleaseActualDevice in qemuBuildCommandLine. This patch does just that. 2011-11-21 Michal Privoznik qemu: Copy console definition from serial Now, when we support multiple consoles per domain, the vm->def->console[0] can still remain an alias for vm->def->serial[0]; However, we need to copy it's source definition as well otherwise we'll regress on virDomainOpenConsole. 2011-11-21 Daniel P. Berrange Fix distribution of .syms files for previous commit * src/Makefile.am: Remove libvirt_bridge.syms and libvirt_macvtap.syms from EXTRA_DIST. Add libvirt_dbus.syms 2011-11-21 Daniel P. Berrange Fix build on Mingw32 wrt export of virNetServerGetDBusConn Mingw32 complains if you request export of a symbol which does not in fact exist. * src/libvirt_bridge.syms, src/libvirt_macvtap.syms: Delete obsolete files * src/libvirt_private.syms: Remove virNetServerGetDBusConn * src/libvirt_dbus.syms: Add virNetServerGetDBusConn 2011-11-21 Osier Yang storage: Skips backingStore of virtual snapshot lv lvs outputs "[$lvname_vorigin]" for the virtual snapshot lv (created with "--virtualsize"), and the original device pointed by "$lvname_vorigin" is just for lvm internal use, one should never use it. Per lvm's nameing rules, "[" is not valid as part of the vg/lv name. (man 8 lvm). VALID NAMES The following characters are valid for VG and LV names: a-z A-Z 0-9 + _ . - VG and LV names cannot begin with a hyphen. There are also various reserved names that are used internally by lvm that can not be used as LV or VG names. A VG cannot be called anything that exists in /dev/ at the time of creation, nor can it be called '.' or '..'. A LV cannot be called '.' '..' 'snapshot' or 'pvmove'. The LV name may also not con‐ tain the strings '_mlog' or '_mimage' So we can skip the set the lv's backingStore by checking if the name begins with a "[". 2011-11-19 Stefan Berger Add documentation for VLAN filtering support Add documentation for the VLAN filtering support. Describe the XML attributes that are supported. Add test cases for VLAN traffic filtering This patch adds a few test cases for the XML parsing of VLAN filtering nodes. Add support for VLAN filtering This patch adds support for filtering of VLAN (802.1Q) traffic to the parser and makes us of the ebtables support for VLAN filtering. This code now enables the filtering of traffic in chains with prefix 'vlan'. 2011-11-19 Jim Fehlig Don't copy sexpr node value that is an empty string Xen4.1 initializes some unspecified sexpr config items to an empty string, unlike previous Xen versions that would leave the item unset. E.g. the kernel item for an HVM guest (non-direct kernel boot): Xen4.0 and earlier ... (image (hvm (kernel ) ... Xen4.1 ... (image (hvm (kernel '') ... The empty string for kernel causes some grief in subsequent parsing where existence of specified kernel is checked, e.g. if (!def->os.kernel) ... This patch solves the problem in sexpr_node_copy() by not copying a node containing an empty string. 2011-11-18 Eric Blake tests: avoid xend ABRT crash report I installed the xen development packages on my non-Xen F16 machine in order to compile-test xen code and ensure we don't break things on that front, but being a non-xen machine, /usr/sbin/xend is obviously not running. Unfortunately, xen-4.1.2-1.fc16 has a bug where merely trying to probe xend status on a non-xen kernel causes xend to issue an ABRT crash report: https://bugzilla.redhat.com/show_bug.cgi?id=728696 Even though libvirt (correctly) skips the test, the xend crash report is unnecessary noise. Fix this by first filtering out non-xen kernels even before attempting to probe xend. The test still runs and passes on a RHEL 5 xen kernel after this patch. * tests/reconnect.c (mymain): Skip xend probe on non-xen kernel. * tests/statstest.c (mymain): Likewise. 2011-11-18 Eric Blake build: fix compile error with no macvtap Since commit 6ec8288a, compilation has failed on RHEL 5: util/virnetdevmacvlan.c:672: error: conflicting types for 'virNetDevMacVLanCreateWithVPortProfile' * src/util/virnetdevmacvlan.c (virNetDevMacVLanCreateWithVPortProfile): Add missing parameter. 2011-11-18 Hu Tao enable cgroup cpuset by default This prepares for subsequent patches which introduce dependence on cgroup cpuset. Enable cgroup cpuset by default so users don't have to modify configuration file before encountering a cpuset error. 2011-11-18 Eric Blake build: fix accidental POTFILES.in regression The original patch for commit 4789fb2 considered renaming a file, then backed out the name change, but forgot to back out the POTFILES.in change, resulting in 'make syntax-check' failure. 2011-11-18 Eric Blake tests: test recent hash addition Excercise the new hash API, to ensure we avoid regressions. * tests/hashtest.c (testHashGetItems): New test. 2011-11-18 Stefan Berger Add test cases for parsing of list values This patch adds test cases for parsing of parameters with multiple occurrances of the same name. 2011-11-18 Stefan Berger Extend NWFilter parameter parser to cope with lists of values This patch modifies the NWFilter parameter parser to support multiple elements with the same name and to internally build a list of items. An example of the XML looks like this: The list of values is then stored in the newly introduced data type virNWFilterVarValue. The XML formatter is also adapted to print out all items in alphabetical order sorted by 'name'. This patch also fixes a bug in the XML schema on the way. 2011-11-18 Stefan Berger Create rules for each member of a list This patch extends the NWFilter driver for Linux (ebiptables) to create rules for each member of a previously introduced list. If for example an attribute value (internally) looks like this: IP = [10.0.0.1, 10.0.0.2, 10.0.0.3] then 3 rules will be generated for a rule accessing the variable 'IP', one for each member of the list. The effect of this is that this now allows for filtering for multiple values in one field. This can then be used to support for filtering/allowing of multiple IP addresses per interface. An iterator is introduced that extracts each member of a list and puts it into a hash table which then is passed to the function creating a rule. For the above example the iterator would cause 3 loops. 2011-11-18 Stefan Berger Rework value part of name-value pairs NWFilters can be provided name-value pairs using the following XML notation: The internal representation currently is so that a name is stored as a string and the value as well. This patch now addresses the value part of it and introduces a data structure for storing a value either as a simple value or as an array for later support of lists. This patch adjusts all code that was handling the values in hash tables and makes it use the new data type. 2011-11-18 Stefan Berger Documentation about chains' priorities, lists of elements etc. This patch adds several aspects of documentation about the network filtering system: - chains, chains' priorities and chains' default priorities - talks about lists of elements, i.e., a variable assigned multiple values (part of already ACK-ed series) - already mentions the vlan, stp and mac chains added later on (https://www.redhat.com/archives/libvir-list/2011-October/msg01238.html) - mentions limitations of vlan filtering (when sent by VM) on Linux systems 2011-11-18 Stefan Berger Add test cases Add test case for the chain names with known prefixes and the chain priority. 2011-11-18 Stefan Berger Interleave jumping into chains with filtering rules in 'root' table The previous patch extends the priority of filtering rules into negative numbers. We now use this possibility to interleave the jumping into chains with filtering rules to for example create the 'root' table of an interface with the following sequence of rules: Bridge chain: libvirt-I-vnet0, entries: 6, policy: ACCEPT -p IPv4 -j I-vnet0-ipv4 -p ARP -j I-vnet0-arp -p ARP -j ACCEPT -p 0x8035 -j I-vnet0-rarp -p 0x835 -j ACCEPT -j DROP The '-p ARP -j ACCEPT' rule now appears between the jumps. Since the 'arp' chain has been assigned priority -700 and the 'rarp' chain -600, the above ordering can now be achieved with the following rule: This patch now sorts the commands generating the above shown jumps into chains and interleaves their execution with those for generating rules. 2011-11-18 Stefan Berger Extend rule priorities into negative numbers So far rules' priorities have only been valid in the range [0,1000]. Now I am extending their priority into the range [-1000, 1000] for subsequently being able to sort rules and the access of (jumps into) chains following priorities. 2011-11-18 Stefan Berger Enable chains with names having a known prefix This patch enables chains that have a known prefix in their name. Known prefixes are: 'ipv4', 'ipv6', 'arp', 'rarp'. All prefixes are also protocols that can be evaluated on the ebtables level. Following the prefix they will be automatically connected to an interface's 'root' chain and jumped into following the protocol they evaluate, i.e., a table 'arp-xyz' will be accessed from the root table using ebtables -t nat -A -p arp -j I--arp-xyz thus generating a 'root' chain like this one here: Bridge chain: libvirt-O-vnet0, entries: 5, policy: ACCEPT -p IPv4 -j O-vnet0-ipv4 -p ARP -j O-vnet0-arp -p 0x8035 -j O-vnet0-rarp -p ARP -j O-vnet0-arp-xyz -j DROP where the chain 'arp-xyz' is accessed for filtering of ARP packets. 2011-11-18 Stefan Berger Extend the filter XML to support priorities of chains This patch extends the filter XML to support priorities of chains in the XML. An example would be: [...] The permitted values for priorities are [-1000, 1000]. By setting the priority of a chain the order in which it is accessed from the interface root chain can be influenced. 2011-11-18 Stefan Berger Use the actual names of chains in data structure Use the name of the chain rather than its type index (enum). This pushes the later enablement of chains with user-given names into the XML parser. For now we still only allow those names that are well known ('root', 'arp', 'rarp', 'ipv4' and 'ipv6'). 2011-11-18 Stefan Berger Use scripting for cleaning and renaming of chains Use scripts for the renaming and cleaning up of chains. This allows us to get rid of some of the code that is only capable of renaming and removing chains whose names are hardcoded. A shell function 'collect_chains' is introduced that is given the name of an ebtables chain and then recursively determines the names of all chains that are accessed from this chain and its sub-chains using 'jumps'. The resulting list of chain names is then used to delete all the found chains by first flushing and then deleting them. The same function is also used for renaming temporary filters to their final names. I tested this with the bash and dash as script interpreters. 2011-11-18 Stefan Berger Make filter creation in root table more flexible Use the previously introduced chain priorities to sort the chains for access from an interface's 'root' table and have them created in the proper order. This gets rid of a lot of code that was previously creating the chains in a more hardcoded way. To determine what protocol a filter is used for evaluation do prefix- matching, i.e., the filter 'arp' is used to filter for the 'arp' protocol, 'ipv4' for the 'ipv4' protocol and 'arp-xyz' will also be used to filter for the 'arp' protocol following the prefix 'arp' in its name. 2011-11-18 Stefan Berger Introduce an internal priority for chains For better handling of the sorting of chains introduce an internally used priority. Use a lookup table to store the priorities. For now their actual values do not matter just that the values cause the chains to be properly sorted through changes in the following patches. However, the values are chosen as negative so that once they are sorted along with filtering rules (whose priority may only be positive for now) they will always be instantiated before them (lower values cause instantiation before higher values). This is done to maintain backwards compatibility. Add function to get hash table's key/value pairs Add a function to the virHashTable for getting an array of the hash table's key-value pairs and have the keys (optionally) sorted. 2011-11-18 Daniel P. Berrange Add support for systemd init service This patch adds support for a systemd init service for libvirtd and libvirt-guests. The libvirtd.service is *not* written to use socket activation, since we want libvirtd to start on boot so it can do guest auto-start. The libvirt-guests.service is pretty lame, just exec'ing the original init script for now. Ideally we would factor out the functionality, into some shared tool. Instead of ./configure --with-init-script=redhat You can now do ./configure --with-init-script=systemd Or better still: ./configure --with-init-script=systemd+redhat We can also now support install of the upstart init script * configure.ac: Add systemd, and systemd+redhat options to --with-init-script option * daemon/Makefile.am: Install systemd services * daemon/libvirtd.sysconf: Add note about unused env variable with systemd * daemon/libvirtd.service.in: libvirtd systemd service unit * libvirt.spec.in: Add scripts to installing systemd services and migrating from legacy init scripts * tools/Makefile.am: Install systemd services * tools/libvirt-guests.init.sh: Rename to tools/libvirt-guests.init.in * tools/libvirt-guests.service.in: systemd service unit 2011-11-18 Daniel P. Berrange Add support for interfaces with type=direct to LXC Support creation of macvlan devices for LXC containers. Do not allow setting of bandwidth controls or vport profiles due to the complication that there is no host side visible device to work with. * src/lxc/lxc_driver.c: Support type=direct interfaces 2011-11-18 Daniel P. Berrange Allow creation of plain macvlan devices Update virNetDevMacVLanCreateWithVPortProfile to allow creation of plain macvlan devices, as well as macvtap devices. The former is useful for LXC containers * src/qemu/qemu_command.c: Explicitly request a macvtap device * src/util/virnetdevmacvlan.c, src/util/virnetdevmacvlan.h: Add new flag to allow switching between macvlan and macvtap creation 2011-11-18 Daniel P. Berrange Refactor LXC network setup to allow future enhancements The current lxcSetupInterfaces() method directly performs setup of the bridge devices. Since it will shortly need to also create macvlan devices, move the bridge related code into a separate method * src/lxc/lxc_driver.c: Split lxcSetupInterfaces() to create a new lxcSetupInterfaceBridge() 2011-11-18 Daniel P. Berrange Add missing 'const' annotations for internal domain conf helpers The virDomainNetGetActualBridgeName and virDomainNetGetActualDirectDev methods both return strings that point to data in the virDomainDefPtr struct, and should therefore not be freed. The return values should thus be 'const char *' not 'char *'. * src/conf/domain_conf.c, src/conf/domain_conf.h: Mark const * src/network/bridge_driver.c: Update to use a const char * 2011-11-18 Daniel P. Berrange Fix up ordering of private symbols file Fix ordering of symbols after re-arranging network device management API source files * src/libvirt_private.syms: Fix ordering 2011-11-18 Daniel P. Berrange Move ifaceMacvtapLinkDump and ifaceGetNthParent functions Move the ifaceMacvtapLinkDump and ifaceGetNthParent functions into virnetdevvportprofile.c since they are specific to that code. This avoids polluting the headers with the Linux specific netlink data types * src/util/interface.c, src/util/interface.h: Move ifaceMacvtapLinkDump and ifaceGetNthParent functions and delete remaining file * src/util/virnetdevvportprofile.c: Add ifaceMacvtapLinkDump and ifaceGetNthParent functions * src/network/bridge_driver.c, src/nwfilter/nwfilter_gentech_driver.c, src/nwfilter/nwfilter_learnipaddr.c, src/util/virnetdevmacvlan.c: Remove include of interface.h 2011-11-18 Daniel P. Berrange Move functions for dealing with physical/virtual devices Move virNetDevIsVirtualFunction, virNetDevGetVirtualFunctionIndex and virNetDevGetPhysicalFunction to virnetdev.c * src/util/interface.c, src/util/interface.h, src/util/virnetdev.c, src/util/virnetdev.h: Move APIs 2011-11-18 Daniel P. Berrange Rename APIs for dealing with virtual/physical functions Rename ifaceIsVirtualFunction to virNetDevIsVirtualFunction, ifaceGetVirtualFunctionIndex to virNetDevGetVirtualFunctionIndex and ifaceGetPhysicalFunction to virNetDevGetPhysicalFunction * src/util/interface.c, src/util/interface.h: Rename APIs * src/util/virnetdevvportprofile.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevValidateConfig to virnetdev.c * src/util/interface.c, src/util/interface.h: Remove virNetDevValidateConfig * src/util/virnetdev.c, src/util/virnetdev.h: Add virNetDevValidateConfig 2011-11-18 Daniel P. Berrange Rename ifaceCheck to virNetDevValidateConfig Rename the ifaceCheck method to virNetDevValidateConfig and change so that it always raises an error and returns -1 on error. * src/util/interface.c, src/util/interface.h: Rename ifaceCheck to virNetDevValidateConfig * src/nwfilter/nwfilter_gentech_driver.c, src/nwfilter/nwfilter_learnipaddr.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevGetIPv4Address to virnetdev.c Move the virNetDevGetIPv4Address function to virnetdev.c * util/interface.c, util/interface.h: Remove virNetDevGetIPv4Address * util/virnetdev.c, util/virnetdev.h: Add virNetDevGetIPv4Address 2011-11-18 Daniel P. Berrange Rename ifaceGetIPAddress to virNetDevGetIPv4Address To match up with the existing virNetDevSetIPv4Address, rename ifaceGetIPAddress to virNetDevGetIPv4Address * util/interface.h, util/interface.c: Rename API * network/bridge_driver.c: Update for API rename 2011-11-18 Daniel P. Berrange Move virNetDevGetIndex & virNetDevGetVLanID to virnetdev.c Move virNetDevGetIndex & virNetDevGetVLanID to virnetdev.c to suit their functional purpose * util/interface.c, util/interface.h: Remove virNetDevGetIndex & virNetDevGetVLanID * util/virnetdev.c, util/virnetdev.h: Add virNetDevGetIndex & virNetDevGetVLanID 2011-11-18 Daniel P. Berrange Rename ifaceGetIndex and ifaceGetVLAN Rename the ifaceGetIndex method to virNetDevGetIndex and ifaceGetVlanID to virNetDevGetVLanID. Also change the error reporting behaviour to always raise errors and return -1 on failure * util/interface.c, util/interface.h: Rename ifaceGetIndex and ifaceGetVLAN * nwfilter/nwfilter_gentech_driver.c, nwfilter/nwfilter_learnipaddr.c, nwfilter/nwfilter_learnipaddr.c, util/virnetdevvportprofile.c: Update for API renames and error handling changes 2011-11-18 Daniel P. Berrange Move MAC address replacement functions to virnetdev.c Move virNetDevReplaceMacAddress and virNetDevRestoreMacAddress to the virnetdev.c file where they naturally belong * util/interface.c, util/interface.h: Remove virNetDevReplaceMacAddress and virNetDevRestoreMacAddress * util/virnetdev.c, util/virnetdev.h: Add virNetDevReplaceMacAddress and virNetDevRestoreMacAddress 2011-11-18 Daniel P. Berrange Rename interface MAC address replacement APIs Rename ifaceReplaceMacAddress to virNetDevReplaceMacAddress and ifaceRestoreMacAddress to virNetDevRestoreMacAddress. * util/interface.c, util/interface.h, util/virnetdevmacvlan.c: Rename APIs 2011-11-18 Daniel P. Berrange Move the low level macvlan creation APIs Move the low level macvlan creation APIs into the virnetdevmacvlan.c file where they more naturally belong * util/interface.c, util/interface.h: Remove virNetDevMacVLanCreate and virNetDevMacVLanDelete * util/virnetdevmacvlan.c, util/virnetdevmacvlan.h: Add virNetDevMacVLanCreate and virNetDevMacVLanDelete 2011-11-18 Daniel P. Berrange Rename low level macvlan creation APIs Rename ifaceMacvtapLinkAdd to virNetDevMacVLanCreate and ifaceLinkDel to virNetDevMacVLanDelete. Strictly speaking the latter isn't restricted to macvlan devices, but that's the only use libvirt has for it. * util/interface.c, util/interface.h, util/virnetdevmacvlan.c: Rename APIs 2011-11-18 Daniel P. Berrange Rename high level macvlan creation APIs Rename virNetDevMacVLanCreate to virNetDevMacVLanCreateWithVPortProfile and virNetDevMacVLanDelete to virNetDevMacVLanDeleteWithVPortProfile To make way for renaming the other macvlan creation APIs in interface.c * util/virnetdevmacvlan.c, util/virnetdevmacvlan.h, qemu/qemu_command.c, qemu/qemu_hotplug.c, qemu/qemu_process.c: Rename APIs 2011-11-18 Daniel P. Berrange Rename and split the macvtap.c file Rename the macvtap.c file to virnetdevmacvlan.c to reflect its functionality. Move the port profile association code out into virnetdevvportprofile.c. Make the APIs available unconditionally to callers * src/util/macvtap.h: rename to src/util/virnetdevmacvlan.h, * src/util/macvtap.c: rename to src/util/virnetdevmacvlan.c * src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h: Pull in vport association code * src/Makefile.am, src/conf/domain_conf.h, src/qemu/qemu_conf.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c: Update include paths & remove conditional compilation 2011-11-18 Daniel P. Berrange Rename Macvtap management APIs In preparation for code re-organization, rename the Macvtap management APIs to have the following patterns virNetDevMacVLanXXXXX - macvlan/macvtap interface management virNetDevVPortProfileXXXX - virtual port profile management * src/util/macvtap.c, src/util/macvtap.h: Rename APIs * src/conf/domain_conf.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_command.h, src/qemu/qemu_driver.c, src/qemu/qemu_hotplug.c, src/qemu/qemu_migration.c, src/qemu/qemu_process.c, src/qemu/qemu_process.h: Update for renamed APIs 2011-11-18 Daniel P. Berrange Fix use of uninitialized variable in QEMU driver 2011-11-17 Bharata B Rao qemu: Generate -numa option Add routines to generate -numa QEMU command line option based on ... XML specifications. 2011-11-17 Bharata B Rao XML definitions for guest NUMA and parsing routines This patch adds XML definitions for guest NUMA specification and contains routines to parse the same. The guest NUMA specification looks like this: ... ... 2011-11-17 Eric Blake snapshot: refuse to generate names for non-regular backing files For whatever reason, the kernel allows you to create a regular file named /dev/sdc.12345; although this file will disappear the next time devtmpfs is remounted. If you let libvirt generate the name of the external snapshot for a disk image originally using the block device /dev/sdc, then the domain will be rendered unbootable once the qcow2 file is lost on the next devtmpfs remount. In this case, the user should have used 'virsh snapshot-create --xmlfile' or 'virsh snapshot-create-as --diskspec' to specify the name for the qcow2 file in a sane location, rather than relying on libvirt generating a name that is most likely to be wrong. We can help avoid naive mistakes by enforcing that the user provide the external name for any backing file that is not a regular file. * src/conf/domain_conf.c (virDomainSnapshotAlignDisks): Only generate names if backing file exists as regular file. Reported by MATSUDA Daiki. 2011-11-16 Jim Fehlig Fix build with polkit0 I missed adding virNetServerGetDBusConn() to libvirtd_private.syms in commit b8adfcc6, which didn't cause a problem in 0.9.6 but results in this build error in 0.9.7 libvirtd-remote.o: In function `remoteDispatchAuthPolkit': remote.c:(.text+0x188dd): undefined reference to `virNetServerGetDBusConn' 2011-11-16 Jim Fehlig Revert commit 0f590c62 As noted by Daniel Berrange [1], the proper fix for the older PolicyKit build issue is to add virNetServerGetDBusConn to libvirt_private.syms. Revert unnecessary changes to daemon/Makefile.am [1] https://www.redhat.com/archives/libvir-list/2011-November/msg00852.html 2011-11-16 Laine Stump virsh: add iface-bridge and iface-unbridge commands One of the top questions by libvirt users is how to create a host bridge device so that guests can be directly on the physical network. There are several example documents that explain how to do this manually, but following them often results in confusion and failure. virt-manager does a good job of creating a bridge based on an existing network device, but not everyone wants to use virt-manager. This patch adds a new command, iface-bridge that makes it just about as simple as possible to create a new bridge device based on an existing ethernet/vlan/bond device (including associating IP configuration with the bridge rather than the now-attached device), and start that new bridge up ready for action, eg: virsh iface-bridge eth0 br0 For symmetry's sake, it also adds a command to remove a device from a bridge, restoring the IP config to the now-unattached device: virsh iface-unbridge br0 (I had a short debate about whether to do "iface-unbridge eth0" instead, but that would involve searching through all bridge devices for the one that contained eth0, which seems like a bit too much trouble). NOTE: These two commands require that the netcf library be available on the host. Hopefully this will provide some extra incentive for people using suse, debian, ubuntu, and other similar systems to polish up (and push downstream) the ports to those distros recently pushed to the upstream netcf repo by Dan Berrange. Anyone interested in helping with that effort in any way should join the netcf-devel mailing list (subscription info at https://fedorahosted.org/mailman/listinfo/netcf-devel) During creation of the bridge, it's possible to specify whether or not the STP protocol should be started up on the bridge and, if so, how many seconds the bridge should squelch traffic from newly added devices while learning new topology (defaults are stp='on' and delay='0', which seems to usually work best for bridges used in the context of libvirt guests). There is also an option to not immediately start the bridge (and a similar option to not immediately start the un-attached device after destroying the bridge. Default is to start the new device, because in the case of iface-unbridge not starting is strongly discouraged as it will leave the system with no network connectivity on that interface (because it's necessary to destroy/undefine the bridge device before the unattached device can be defined), and it seemed better to make the option for iface-bridge behave consistently. NOTE TO THOSE TRYING THESE COMMANDS FOR THE FIRST TIME: to guard against any "unexpected" change to configuration, it is advisable to issue an "virsh iface-begin" command before starting any interface config changes, and "virsh iface-commit" only after you've verified that everything is working as you expect. If something goes wrong, you can always run "virsh iface-rollback" or reboot the system (which should automatically do iface-rollback). Aside from adding the code for these two functions, and the two entries into the command table, the only other change to virsh.c was to add the option name to vshCommandOptInterfaceBy(), because the iface-unbridge command names its interface option as "bridge". virsh.pod has also been updated with short descriptions of these two new commands. 2011-11-16 Hu Tao fix a bug in remoteSerializeTypedParameters This is a fatal typo believed to be very likely to happen when using both i and j at the same time for indexing. 2011-11-16 Daniel P. Berrange Don't return a fatal error if receiving unexpected stream data Due to the asynchronous nature of streams, we might continue to receive some stream packets from the server even after we have shutdown the stream on the client side. These should be discarded silently, rather than raising an error in the RPC layer. * src/rpc/virnetclient.c: Discard stream data silently 2011-11-16 Daniel P. Berrange Fix handling of stream EOF Very occasionally the sequence of events from poll would result in getting a HANGUP on its own, instead of a HANGUP+READABLE at the same time. In the former case we would send back an error event to the client, but never send the empty packet to indicate EOF. 2011-11-16 Daniel P. Berrange Allow non-blocking message sending on virNetClient Add a new virNetClientSendNonBlock which returns 2 on full send, 1 on partial send, 0 on no send, -1 on error If a partial send occurs, then a subsequent call to any of the virNetClientSend* APIs will finish any outstanding I/O. TODO: the virNetClientEvent event handler could be used to speed up completion of partial sends if an event loop is present. * src/rpc/virnetsocket.h, src/rpc/virnetsocket.c: Add new virNetSocketHasPendingData() API to test for cached data pending send. * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add new virNetClientSendNonBlock() API to send non-blocking API 2011-11-16 Daniel P. Berrange Refactor code for enabling/disabling I/O callback in remote client * src/rpc/virnetclient.c: Add helper for setting I/O callback events 2011-11-16 Daniel P. Berrange Split virNetClientSend into 2 methods Stop multiplexing virNetClientSend for two different purposes, instead add virNetClientSendWithReply and virNetClientSendNoReply * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Replace virNetClientSend with virNetClientSendWithReply and virNetClientSendNoReply * src/rpc/virnetclientprogram.c, src/rpc/virnetclientstream.c: Update for new API names 2011-11-16 Daniel P. Berrange Refactor code for passing the buck in the remote client Remove some duplication by pulling the code for passing the buck out into a helper method * src/rpc/virnetclient.c: Introduce virNetClientIOEventLoopPassTheBuck 2011-11-16 Daniel P. Berrange Explicitly track whether the buck is held in remote client Instead of inferring whether the buck is held from the waitDispatch pointer, use an explicit 'bool haveTheBuck' field * src/rpc/virnetclient.c: Explicitly track the buck 2011-11-16 Daniel P. Berrange Remove all linked list handling from remote client event loop Directly messing around with the linked list is potentially dangerous. Introduce some helper APIs to deal with list manipulating the list * src/rpc/virnetclient.c: Create linked list handlers 2011-11-16 Eli Qiao util: Add netdev helper functions to private symbols 2011-11-16 Sage Weil qemu/rbd: improve rbd device specification This improves the support for qemu rbd devices by adding support for a few key features (e.g., authentication) and cleaning up the way in which rbd configuration options are passed to qemu. An member of the disk source xml specifies how librbd should authenticate. The username attribute is the Ceph/RBD user to authenticate as. The usage or uuid attributes specify which secret to use. Usage is an arbitrary identifier local to libvirt. The old RBD support relied on setting an environment variable to communicate information to qemu/librbd. Instead, pass those options explicitly to qemu. Update the qemu argument parsing and tests accordingly. 2011-11-15 Jim Fehlig Fix build with polkit0 I missed adding libvirt_driver_remote.la to libvirtd_LDADD in commit b8adfcc6, which didn't cause a problem in 0.9.6 but results in this build error in 0.9.7 libvirtd-remote.o: In function `remoteDispatchAuthPolkit': remote.c:(.text+0x188dd): undefined reference to `virNetServerGetDBusConn' 2011-11-15 Stefan Berger Fix strchr call triggering gcc 4.3 & 4.4 bug Replacing the strchr call with two variables through a strstr call. Calling strchr with two variables triggers a gcc 4.3/4.4 bug when used in combination with -Wlogical-op and at least -O1. maint: fix build include stdint.h to fix the build 2011-11-15 Daniel P. Berrange Remove ifaceSetMac and ifaceGetMac APIs The ifaceSetMac and ifaceGetMac APIs duplicate the functionality of the virNetDevSetMAC and virNetDevGetMAC APIs, but returning errno's instead of raising errors. * src/util/interface.c, src/util/interface.h: Remove ifaceSetMac and ifaceGetMac APIs, adjusting callers for new error behaviour 2011-11-15 Daniel P. Berrange Remove ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp APIs The ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp APIs can be replaced with calls to virNetDevSetOnline and virNetDevIsOnline * src/util/interface.c, src/util/interface.h: Delete ifaceUp, ifaceDown, ifaceCtrl & ifaceIsUp * src/nwfilter/nwfilter_gentech_driver.c, src/util/macvtap.c: Update to use virNetDevSetOnline and virNetDevIsOnline 2011-11-15 Daniel P. Berrange Move LXC veth.c code into shared utility APIs Move the virNetDevSetName and virNetDevSetNamespace APIs out of LXC's veth.c and into virnetdev.c. Move the remaining content of the file to src/util/virnetdevveth.c * src/lxc/veth.c: Rename to src/util/virnetdevveth.c * src/lxc/veth.h: Rename to src/util/virnetdevveth.h * src/util/virnetdev.c, src/util/virnetdev.h: Add virNetDevSetName and virNetDevSetNamespace * src/lxc/lxc_container.c, src/lxc/lxc_controller.c, src/lxc/lxc_driver.c: Update include paths 2011-11-15 Daniel P. Berrange Rename the LXC veth management APIs and delete duplicated APIs The src/lxc/veth.c file contains APIs for managing veth devices, but some of the APIs duplicate stuff from src/util/virnetdev.h. Delete thed duplicate APIs and rename the remaining ones to follow virNetDevVethXXXX * src/lxc/veth.c, src/lxc/veth.h: Rename APIs & delete duplicates * src/lxc/lxc_container.c, src/lxc/lxc_controller.c, src/lxc/lxc_driver.c: Update for API renaming 2011-11-15 Daniel P. Berrange Split src/util/network.{c,h} into 5 pieces The src/util/network.c file is a dumping ground for many different APIs. Split it up into 5 pieces, along functional lines - src/util/virnetdevbandwidth.c: virNetDevBandwidth type & helper APIs - src/util/virnetdevvportprofile.c: virNetDevVPortProfile type & helper APIs - src/util/virsocketaddr.c: virSocketAddr and APIs - src/conf/netdev_bandwidth_conf.c: XML parsing / formatting for virNetDevBandwidth - src/conf/netdev_vport_profile_conf.c: XML parsing / formatting for virNetDevVPortProfile * src/util/network.c, src/util/network.h: Split into 5 pieces * src/conf/netdev_bandwidth_conf.c, src/conf/netdev_bandwidth_conf.h, src/conf/netdev_vport_profile_conf.c, src/conf/netdev_vport_profile_conf.h, src/util/virnetdevbandwidth.c, src/util/virnetdevbandwidth.h, src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h, src/util/virsocketaddr.c, src/util/virsocketaddr.h: New pieces * daemon/libvirtd.h, daemon/remote.c, src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/conf/nwfilter_conf.h, src/esx/esx_util.h, src/network/bridge_driver.c, src/qemu/qemu_conf.c, src/rpc/virnetsocket.c, src/rpc/virnetsocket.h, src/util/dnsmasq.h, src/util/interface.h, src/util/iptables.h, src/util/macvtap.c, src/util/macvtap.h, src/util/virnetdev.h, src/util/virnetdevtap.c, tools/virsh.c: Update include files 2011-11-15 Daniel P. Berrange Fix error reporting in port profile parsing/formatting APIs The virtual port profile parsing/formatting APIs do not correctly handle unknown profile type strings/numbers. They behave as a no-op, instead of raising an error * src/util/network.c, src/util/network.h: Fix error handling of port profile APIs * src/conf/domain_conf.c, src/conf/network_conf.c: Update for API changes 2011-11-15 Daniel P. Berrange Rename virVirtualPortProfileParams & APIs Rename the virVirtualPortProfileParams struct to be virNetDevVPortProfile, and rename the APIs to match this prefix. * src/util/network.c, src/util/network.h: Rename port profile APIs * src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/network/bridge_driver.c, src/qemu/qemu_hotplug.c, src/util/macvtap.c, src/util/macvtap.h: Update for renamed APIs/structs 2011-11-14 Stefan Berger maint: Add Michael Wood as an author Add Michael Wood as an author. 2011-11-14 Michael Wood PATCH: Fix build without MACVTAP Hi Commit c31d23a78715f1144c73862c46ab0436de8b5e85 removed the "conn" parameter from qemuPhysIfaceConnect(), but it's still used if WITH_MACVTAP is false. Also, it's still mentioned in the comment above the function: /** * qemuPhysIfaceConnect: * @def: the definition of the VM (needed by 802.1Qbh and audit) * @conn: pointer to virConnect object * @driver: pointer to the qemud_driver * @net: pointer to he VM's interface description with direct device type * @qemuCaps: flags for qemu * * Returns a filedescriptor on success or -1 in case of error. */ int qemuPhysIfaceConnect(virDomainDefPtr def, struct qemud_driver *driver, virDomainNetDefPtr net, virBitmapPtr qemuCaps, enum virVMOperationType vmop) { int rc; #if WITH_MACVTAP [...] #else (void)def; (void)conn; (void)net; (void)qemuCaps; (void)driver; (void)vmop; qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("No support for macvtap device")); rc = -1; #endif return rc; } -- Michael Wood From f4fc43b4111a4c099395c55902e497b8965e2b53 Mon Sep 17 00:00:00 2001 From: Michael Wood Date: Sat, 12 Nov 2011 13:37:53 +0200 Subject: [PATCH] Fix build without MACVTAP. 2011-11-14 Guido Günther storage: forbid rebuilding existing disk storage pools which would blow away all volumes. Honor VIR_STORAGE_POOL_BUILD_OVERWRITE to force a rebuild. This was caught by libvirt-tck's storage/110-disk-pool.t. 2011-11-12 Eric Blake API: add trivial qemu support for VIR_TYPED_PARAM_STRING Qemu will be the first driver to make use of a typed string in the next round of additions. Separate out the trivial addition. * src/qemu/qemu_driver.c (qemudSupportsFeature): Advertise feature. (qemuDomainGetBlkioParameters, qemuDomainGetMemoryParameters) (qemuGetSchedulerParametersFlags, qemudDomainBlockStatsFlags): Allow typed strings flag where trivially supported. 2011-11-12 Eric Blake API: remote support for VIR_TYPED_PARAM_STRING Send and receive string typed parameters across RPC. This also completes the back-compat mentioned in the previous patch - the only time we have an older client talking to a newer server is if RPC is in use, so filtering out strings during RPC prevents returning an unknown type to the older client. * src/remote/remote_protocol.x (remote_typed_param_value): Add another union value. * daemon/remote.c (remoteDeserializeTypedParameters): Handle strings on rpc. (remoteSerializeTypedParameters): Likewise; plus filter out strings when replying to older clients. Adjust callers. * src/remote/remote_driver.c (remoteFreeTypedParameters) (remoteSerializeTypedParameters) (remoteDeserializeTypedParameters): Handle strings on rpc. * src/rpc/gendispatch.pl: Properly clean up typed arrays. * src/remote_protocol-structs: Update. Based on an initial patch by Hu Tao, with feedback from Daniel P. Berrange. 2011-11-12 Eric Blake API: add VIR_TYPED_PARAM_STRING This allows strings to be transported between client and server in the context of name-type-value virTypedParameter functions. For compatibility, o new clients will not send strings to old servers, based on a feature check o new servers will not send strings to old clients without the flag VIR_TYPED_PARAM_STRING_OKAY; this will be enforced at the RPC layer in the next patch, so that drivers need not worry about it in general. The one exception is that virDomainGetSchedulerParameters lacks a flags argument, so it must not return a string; drivers that forward that function on to virDomainGetSchedulerParametersFlags will have to pay attention to the flag. o the flag VIR_TYPED_PARAM_STRING_OKAY is set automatically, based on a feature check (so far, no driver implements it), so clients do not have to worry about it Future patches can then enable the feature on a per-driver basis. This patch also ensures that drivers can blindly strdup() field names (previously, a malicious client could stuff 80 non-NUL bytes into field and cause a read overrun). * src/libvirt_internal.h (VIR_DRV_FEATURE_TYPED_PARAM_STRING): New driver feature. * src/libvirt.c (virTypedParameterValidateSet) (virTypedParameterSanitizeGet): New helper functions. (virDomainSetMemoryParameters, virDomainSetBlkioParameters) (virDomainSetSchedulerParameters) (virDomainSetSchedulerParametersFlags) (virDomainGetMemoryParameters, virDomainGetBlkioParameters) (virDomainGetSchedulerParameters) (virDomainGetSchedulerParametersFlags, virDomainBlockStatsFlags): Use them. * src/util/util.h (virTypedParameterArrayClear): New helper function. * src/util/util.c (virTypedParameterArrayClear): Implement it. * src/libvirt_private.syms (util.h): Export it. Based on an initial patch by Hu Tao, with feedback from Daniel P. Berrange. 2011-11-12 Eli Qiao util: fix compile error on debian Add virnetdev.h,virnetdevbridge.h,virnetdevtap.h to private symbols, since debian linker no longer allows transitive link resolution 2011-11-12 Eric Blake qemu: fix domjobabort regression This reverts commit ef1065cf5ac; see also this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=751900 In qemu 0.15.1 and earlier, during migration to file, the qemu_savevm_state_begin and qemu_savevm_state_iterate methods will both process as much migration data as possible until either 1. The file descriptor returns EAGAIN 2. The bandwidth rate limit is reached If we set the rate limit to ULONG_MAX, test 2 never becomes true. We're passing a plain file descriptor to QEMU and POSIX does not support EAGAIN on regular files / block devices, so test 1 never becomes true either. In the 'virsh save --bypass-cache' case, we pass a pipe instead of a regular fd, but using a pipe adds I/O overhead, so always passing a pipe just so qemu can see EAGAIN doesn't seem nice. The ultimate fix needs to come from qemu - background migration must respect asynchronous abort requests, or else periodically return control to the main handling loop without an EAGAIN and without waiting to hit an insanely large amount of data. But until a version of qemu is fixed to support "unlimited" data rates while still allowing cancellation, the best we can do is avoid the automatic use of unlimited rates from within libvirt (users can still explicitly change the migration rates, if they are aware that they are giving up the ability to cancel a job). Reverting the lone use of QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX is the simplest patch; this slows migration back down to a default 32M/sec cap, but also ensures that the main qemu processing loop will still be responsive to cancellation requests. Hopefully upstream qemu will provide us a means of safely using unlimited speed, including a runtime probe of that capability. * src/qemu/qemu_migration.c (qemuMigrationToFile): Revert attempt to use unlimited migration bandwidth when migrating to file. 2011-11-12 Hu Tao fix two bugs in bridge_driver.c steps to reproduce: 1. having a network xml file(named default.xml) like this one: default c5322c4c-81d0-4985-a363-ad6389780d89 in /etc/libvirt/qemu/networks/, and mark it as autostart: $ ls -l /etc/libvirt/qemu/networks/autostart total 0 lrwxrwxrwx 1 root root 14 Oct 12 14:02 default.xml -> ../default.xml 2. start libvirtd and the device virbr0 is not automatically up. The reason is that the function virNetDevExists is now returns 1 if the device exists, comparing to the former one returns 0 if the device exists. But with only this fix will cause a segmentation fault(the same steps as above) that is fixed by the second chunk of code. 2011-11-11 Eric Blake build: drop useless dirent.h includes * .gnulib: Update to latest, for improved syntax-check. * src/lxc/lxc_container.c (includes): Drop unused include. * src/network/bridge_driver.c: Likewise. * src/node_device/node_device_linux_sysfs.c: Likewise. * src/openvz/openvz_driver.c: Likewise. * src/qemu/qemu_conf.c: Likewise. * src/storage/storage_backend_iscsi.c: Likewise. * src/storage/storage_backend_mpath.c: Likewise. * src/uml/uml_conf.c: Likewise. * src/uml/uml_driver.c: Likewise. 2011-11-11 Eric Blake xenapi: remove unused variable CC libvirt_driver_xenapi_la-xenapi_driver.lo xenapi/xenapi_driver.c: In function 'xenapiDomainGetVcpus': xenapi/xenapi_driver.c:1209:21: error: variable 'cpus' set but not used [-Werror=unused-but-set-variable] * src/xenapi/xenapi_driver.c (xenapiDomainGetVcpus): Silence compiler warning. 2011-11-11 Eric Blake maint: use mailmap, not AUTHORS, for secondary addresses * AUTHORS: Move Stefan's second entry... * .mailmap: ...here. 2011-11-10 Stefan Berger maint: fix make syntax-check Add my 2nd email to the list of AUTHORS to get 'make syntax-check' to pass. 2011-11-10 Stefan Berger Remove code instantiating filters on direct interfaces Remove the code that instantiates network filters on direct type of interfaces. The parser already does not accept it. 2011-11-10 Daniel P. Berrange Disable numactl on ARM architectures too * libvirt.spec.in: Disable numactl on ARM Add libvirt confdir to files section in mingw32 spec * mingw32-libvirt.spec.in: Ensure we own the confdir 2011-11-10 Eric Blake nwfilter: simplify execution of ebiptables scripts It's not worth even worrying about a temporary file, unless we ever expect the script to exceed maximum command-line argument length limits. * src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI): Run the commands as an argument to /bin/sh, rather than worrying about a temporary file. (ebiptablesWriteToTempFile): Delete unused function. 2011-11-10 Eric Blake nwfilter: avoid failure with noexec /tmp If /tmp is mounted with the noexec flag (common on security-conscious systems), then nwfilter will fail to initialize, because we cannot run any temporary script via virRun("/tmp/script"); but we _can_ use "/bin/sh /tmp/script". For that matter, using /tmp risks collisions with other unrelated programs; we already have /var/run/libvirt as a dedicated temporary directory for use by libvirt. * src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesWriteToTempFile): Use internal directory, not /tmp; drop attempts to make script executable; and detect close error. (ebiptablesExecCLI): Switch to virCommand, and invoke the shell to read the script, rather than requiring an executable script. 2011-11-09 Daniel P. Berrange Adjust naming of network device bandwidth management APIs Rename virBandwidth to virNetDevBandwidth, and virRate to virNetDevBandwidthRate. * src/util/network.c, src/util/network.h: Rename bandwidth structs and APIs * src/conf/domain_conf.c, src/conf/domain_conf.h, src/conf/network_conf.c, src/conf/network_conf.h, src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/util/macvtap.c, src/util/macvtap.h, tools/virsh.c: Update for API changes. 2011-11-09 Daniel P. Berrange Santize naming of socket address APIs The socket address APIs in src/util/network.h either take the form virSocketAddrXXX, virSocketXXX or virSocketXXXAddr. Sanitize this so everything is virSocketAddrXXXX, and ensure that the virSocketAddr parameter is always the first one. * src/util/network.c, src/util/network.h: Santize socket address API naming * src/conf/domain_conf.c, src/conf/network_conf.c, src/conf/nwfilter_conf.c, src/network/bridge_driver.c, src/nwfilter/nwfilter_ebiptables_driver.c, src/nwfilter/nwfilter_learnipaddr.c, src/qemu/qemu_command.c, src/rpc/virnetsocket.c, src/util/dnsmasq.c, src/util/iptables.c, src/util/virnetdev.c, src/vbox/vbox_tmpl.c: Update for API renaming 2011-11-09 Daniel P. Berrange Split bridge.h into three separate files Following the renaming of the bridge management APIs, we can now split the source file into 3 corresponding pieces * src/util/virnetdev.c: APIs for any type of network interface * src/util/virnetdevbridge.c: APIs for bridge interfaces * src/util/virnetdevtap.c: APIs for TAP interfaces * src/util/virnetdev.c, src/util/virnetdev.h, src/util/virnetdevbridge.c, src/util/virnetdevbridge.h, src/util/virnetdevtap.c, src/util/virnetdevtap.h: Copied from bridge.{c,h} * src/util/bridge.c, src/util/bridge.h: Split into 3 pieces * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/openvz/openvz_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_conf.h, src/uml/uml_conf.c, src/uml/uml_conf.h, src/uml/uml_driver.c: Update #include directives 2011-11-09 Daniel P. Berrange Remove usage of brctl command line tool Convert the virNetDevBridgeSetSTP and virNetDevBridgeSetSTPDelay to use ioctls instead of spawning brctl. Implement the virNetDevBridgeGetSTP and virNetDevBridgeGetSTPDelay methods which were declared in the header but never existed * src/util/bridge.c: Convert to use bridge ioctls instead of brctl 2011-11-09 Daniel P. Berrange Add an API for retrieving the MAC address of an interface * src/util/bridge.c, src/util/bridge.h: Add virNetDevGetMAC 2011-11-09 Daniel P. Berrange Expose MTU management APIs The MTU management APIs are useful to other code inside libvirt, so should be exposed as non-static APIs. * src/util/bridge.c, src/util/bridge.h: Expose virNetDevSetMTU, virNetDevSetMTUFromDevice & virNetDevGetMTU 2011-11-09 Daniel P. Berrange Turn two int parameters into bools in bridge APIs * src/util/bridge.c, src/util/bridge.h: s/int/bool/ in virNetDevSetOnline and virNetDevBridgeSetSTP 2011-11-09 Daniel P. Berrange Rename all brXXXX APIs to follow new convention The existing brXXX APIs in src/util/bridge.h are renamed to follow one of three different conventions - virNetDevXXX - operations for any type of interface - virNetDevBridgeXXX - operations for bridge interfaces - virNetDevTapXXX - operations for tap interfaces * src/util/bridge.h, src/util/bridge.c: Rename all APIs * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Update for API renaming 2011-11-09 Daniel P. Berrange Make all brXXX APIs raise errors, instead of returning errnos Currently every caller of the brXXX APIs has to store the returned errno value and then raise an error message. This results in inconsistent error messages across drivers, additional burden on the callers and makes the error reporting inaccurate since it is hard to distinguish different scenarios from 1 errno value. * src/util/bridge.c: Raise errors instead of returning errnos * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/qemu/qemu_command.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Remove error reporting code 2011-11-09 Daniel P. Berrange Remove 'brControl' object The bridge management APIs in src/util/bridge.c require a brControl object to be passed around. This holds the file descriptor for the control socket. This extra object complicates use of the API for only a minor efficiency gain, which is in turn entirely offset by the need to fork/exec the brctl command for STP configuration. This patch removes the 'brControl' object entirely, instead opening the control socket & closing it again within the scope of each method. The parameter names for the APIs are also made to consistently use 'brname' for bridge device name, and 'ifname' for an interface device name. Finally annotations are added for non-NULL parameters and return check validation * src/util/bridge.c, src/util/bridge.h: Remove brControl object and update API parameter names & annotations. * src/lxc/lxc_driver.c, src/network/bridge_driver.c, src/uml/uml_conf.h, src/uml/uml_conf.c, src/uml/uml_driver.c, src/qemu/qemu_command.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c: Remove reference to 'brControl' object 2011-11-09 Eric Blake build: fix mingw build of gnulib openpty Commit f7bd00c12 pulled in a gnulib module that fails to compile on mingw. Work around it while waiting for an upstream gnulib fix. * gnulib/local/lib/pty.in.h (openpty): Provide forward declarations of opaque structs not present on mingw. * gnulib/local/lib/openpty.c (openpty): Provide stub for mingw. 2011-11-09 Eric Blake build: allow for local gnulib diffs Commit f7bd00c12 pulled in a gnulib module that fails to compile on mingw. While it would be nice to pull in a newer version of .gnulib that fixes this, it is difficult to backport any .gnulib update to older releases. So, it makes sense to take advantage of gnulib-tool's ability to support local diffs, where we can apply specific diffs in our use of gnulib without waiting for upstream gnulib to pick up those changes, as well as avoiding a wholesale .gnulib update. The existence of local diffs will also make it easier to backport fixes against a tarball (as long as a tarball and libvirt.git share the same .gnulib commit, then the tarball can be patched by applying the same local diffs as a post-release libvirt.git commit, without having to rerun an entire gnulib-tool bootstrap). This patch introduces the framework for supporting local diffs, without actually introducing any. * bootstrap.conf (local_gl_dir): New variable. * autogen.sh (bootstrap_hash): Hash any local diffs, to force a re-bootstrap if just diffs change. * cfg.mk (_update_required): Likewise. 2011-11-09 Alex Jia lxc: free error object to avoid memory leak Detected by Coverity. Leak introduced in commit 9d201a5. * src/lxc/lxc_driver.c: Clean up on failure. 2011-11-09 Alex Jia lxc: free 'ttyFDs' array on return from lxcVmStart Detected by Coverity. Leak introduced in commit 0f31f7b. * src/lxc/lxc_driver.c: Clean up on failure. 2011-11-09 Osier Yang qemu: Fix improper error message for disk detaching s/virDomainDeviceTypeToString/virDomainDiskDeviceTypeToString/ Report by Xu He Jie virsh: Add VSH_OFLAG_EMPTY_OK for attach-disk command As the description of removing CDROM media from http://wiki.libvirt.org/page/QEMUSwitchToLibvirt#eject_DEV Add flag 'VSH_OFLAG_EMPTY_OK' to the option 'source' of attach-disk Then avoid outputting in the XML if 'source' was empty, rather than trusting libvirt domain_conf.c to understand an empty string. 2011-11-08 Daniel Veillard Release of libvirt-0.9.7 * confiure.ac docs/news.html.in libvirt.spec.in: update for release * po/*.po*: update localizations and rebuilt 2011-11-07 Eric Blake build: fix build on platforms without ptsname_r MacOS lacks ptsname_r, and gnulib doesn't (yet) provide it. But we can avoid it altogether, by using gnulib openpty() instead. Note that we do _not_ want the pt_chown module; gnulib uses it only to implement a replacement openpty() if the system lacks both openpty() and granpt(), but all systems that we currently port to either have at least one of openpty() and/or grantpt(), or lack ptys altogether. That is, we aren't porting to any system that requires us to deal with the hassle of installing a setuid pt_chown helper just to use gnulib's ability to provide openpty() on obscure platforms. * .gnulib: Update to latest, for openpty fixes * bootstrap.conf (gnulib_modules): Add openpty, ttyname_r. (gnulib_tool_option_extras): Exclude pt_chown module. * src/util/util.c (virFileOpenTty): Rewrite in terms of openpty and ttyname_r. * src/util/util.h (virFileOpenTtyAt): Delete dead prototype. 2011-11-07 Daniel P. Berrange Add missing defaultConsoleTargetType callback for AppArmour Every instance of virCapsPtr must have the defaultConsoleTargetType field set. * src/security/virt-aa-helper.c: Add defaultConsoleTargetType to virCapsPtr 2011-11-07 Daniel P. Berrange Fix sending/receiving of FDs when stream returns EAGAIN The code calling sendfd/recvfd was mistakenly assuming those calls would never block. They can in fact return EAGAIN and this is causing us to drop the client connection when blocking ocurrs while sending/receiving FDs. Fixing this is a little hairy on the incoming side, since at the point where we see the EAGAIN, we already thought we had finished receiving all data for the packet. So we play a little trick to reset bufferOffset again and go back into polling for more data. * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Update virNetSocketSendFD/RecvFD to return 0 on EAGAIN, or 1 on success * src/rpc/virnetclient.c: Move decoding of header & fds out of virNetClientCallDispatch and into virNetClientIOHandleInput. Handling blocking when sending/receiving FDs * src/rpc/virnetmessage.h: Add a 'donefds' field to track how many FDs we've sent / received * src/rpc/virnetserverclient.c: Handling blocking when sending/receiving FDs 2011-11-04 Eric Blake build: silence compiler warning on BSD Building on 64-bit FreeBSD 8.2 complained about a cast between a pointer and a smaller integer. Going through an intermediate cast shuts up the compiler. * src/util/threads-pthread.c (virThreadSelfID): Silence a warning. 2011-11-04 Eric Blake build: fix linking on BSD While building on FreeBSD (and after fixing a ptsname_r link error), I got this failure: ./.libs/libvirt_util.a(libvirt_util_la-threads.o)(.text+0x240): In function `virThreadCreate': util/threads-pthread.c:185: undefined reference to `pthread_create' It turns out that gnulib used only pthread_join for LIB_PTHREAD, but on FreeBSD, libc provides that (as a stub function); whereas the more complex pthread_create really does require -pthread, which gnulib tracked under [LT]LIBMULTITHREAD. * configure.ac (LIBS): Check LIBMULTITHREAD alongside LIB_PTHREAD. * src/Makefile.am (THREAD_LIBS): New variable. (libvirt_util_la_LIBADD, libvirt_lxc_LDADD): Use it. 2011-11-04 Laine Stump remote: fix mingw32 build tty is initialized, and later set in code that is compiled for all platforms, but is only used in a section that's inside #ifndef WIN32. 2011-11-04 Eric Blake lxc: avoid use-after-free I got this weird failure: error: Failed to start domain simple error: internal error cannot mix caller fds with blocking execution and tracked it down to a use-after-free - virCommandSetOutputFD was storing the address of a stack-local variable, which then went out of scope before the virCommandRun that dereferenced it. Bug introduced in commit 451cfd05 (0.9.2). * src/lxc/lxc_driver.c (lxcBuildControllerCmd): Move log fd registration... (lxcVmStart): ...to caller. 2011-11-04 Daniel P. Berrange Fix naming of constant for disk event All constants related to events should have a prefix of VIR_DOMAIN_EVENT_ * include/libvirt/libvirt.h.in, src/qemu/qemu_domain.c: Rename VIR_DOMAIN_DISK_CHANGE_MISSING_ON_START to VIR_DOMAIN_EVENT_DISK_CHANGE_MISSING_ON_START 2011-11-04 Jiri Denemark conf: Don't free uninitialized pointer This causes libvirtd to crash when both and are used in one domain XML. Introduced by 5fa3d775a9f8cdb3423373eb084219aaf778df11 2011-11-03 Eric Blake build: fix deep VPATH builds I ran into the following build failure: $ mkdir -p build1 build2/a/very/deep/hierarcy $ cd build2/a/very/deep/hierarcy $ ../../../../../configure && make $ cd ../../../../build1 $ ../configure && make ... ../../src/remote/remote_protocol.c:7:55: fatal error: ../../../../../src/remote/remote_protocol.h: No such file or directory Turns out that we were sometimes generating the remote_protocol.c file with information from the VPATH build, which is bad, since any file shipped in the tarball should be idempotent no matter how deep the VPATH build tree that created it. * src/rpc/genprotocol.pl: Don't embed VPATH into generated file. 2011-11-03 Philipp Hahn doc: Add capability. Allow /capabilities/guest/features/deviceboot. 2011-11-03 Eric Blake lxc: use common code for process cleanup Based on a Coverity report - the return value of waitpid() should always be checked, to avoid problems with leaking resources. * src/lxc/lxc_controller.c (lxcControllerRun): Use simpler virPidAbort. 2011-11-03 Daniel P. Berrange Fix default console type setting The default console type may vary based on the OS type. ie a Xen paravirt guests wants a 'xen' console, while a fullvirt guests wants a 'serial' console. A plain integer default console type in the capabilities does not suffice. Instead introduce a callback that is passed the OS type. * src/conf/capabilities.h: Use a callback for default console type * src/conf/domain_conf.c, src/conf/domain_conf.h: Use callback for default console type. Add missing LXC/OpenVZ console types. * src/esx/esx_driver.c, src/libxl/libxl_conf.c, src/lxc/lxc_conf.c, src/openvz/openvz_conf.c, src/phyp/phyp_driver.c, src/qemu/qemu_capabilities.c, src/uml/uml_conf.c, src/vbox/vbox_tmpl.c, src/vmware/vmware_conf.c, src/xen/xen_hypervisor.c, src/xenapi/xenapi_driver.c: Set default console type callback 2011-11-03 Daniel P. Berrange Set aliases for LXC/UML console devices To allow virDomainOpenConsole to access non-primary consoles, device aliases are required to be set. Until now only the QEMU driver has done this. Update LXC & UML to set aliases for any console devices * src/lxc/lxc_driver.c, src/uml/uml_driver.c: Set aliases for console devices 2011-11-03 Daniel P. Berrange Default console target type with no element When no element was set at all, the default console target type was not being honoured * src/conf/domain_conf.c: Set default target type for consoles with no 2011-11-03 Daniel P. Berrange Add support for multiple consoles in LXC Currently the LXC controller only supports setup of a single text console. This is wired up to the container init's stdio, as well as /dev/console and /dev/tty1. Extending support for multiple consoles, means wiring up additional PTYs to /dev/tty2, /dev/tty3, etc, etc. The LXC controller is passed multiple open file handles, one for each console requested. * src/lxc/lxc_container.c, src/lxc/lxc_container.h: Wire up all the /dev/ttyN links required to symlink to /dev/pts/NN * src/lxc/lxc_container.h: Open more container side /dev/pts/NN devices, and adapt event loop to handle I/O from all consoles * src/lxc/lxc_driver.c: Setup multiple host side PTYs 2011-11-03 Daniel P. Berrange Rewrite LXC I/O forwarding to use main event loop The current I/O code for LXC uses a hand crafted event loop to forward I/O between the container & host app, based on epoll to handle EOF on PTYs. This event loop is not easily extensible to add more consoles, or monitor other types of file descriptors. Remove the custom event loop and replace it with a normal libvirt event loop. When detecting EOF on a PTY, disable the event watch on that FD, and fork off a background thread that does a edge-triggered epoll() on the FD. When the FD finally shows new incoming data, the thread re-enables the watch on the FD and exits. When getting EOF from a read() on the PTY, the existing code would do waitpid(WNOHANG) to see if the container had exited. Unfortunately there is a race condition, because even though the process has closed its stdio handles, it might still exist. To deal with this the new event loop uses a SIG_CHILD handler to perform the waitpid only when the container is known to have actually exited. * src/lxc/lxc_controller.c: Rewrite the event loop to use the standard APIs. 2011-11-03 Daniel P. Berrange Fix crash formatting virtio console qemuBuildVirtioSerialPortDevStr was mistakenly accessing the target.name field in the virDomainChrDef object for chardevs belonging to a console. Those chardevs only have port set, and if there's > 1 console, the > 1port number results in trying to access a target.name with address 0x1 * src/qemu/qemu_command.c: Fix target.name handling and make code more robust wrt error reporting * src/qemu/qemu_command.c: Conditionally access target.name 2011-11-03 Daniel P. Berrange Allow multiple consoles per virtual guest While Xen only has a single paravirt console, UML, and QEMU both support multiple paravirt consoles. The LXC driver can also be trivially made to support multiple consoles. This patch extends the XML to allow multiple elements in the XML. It also makes the UML and QEMU drivers support this config. * src/conf/domain_conf.c, src/conf/domain_conf.h: Allow multiple devices * src/lxc/lxc_driver.c, src/xen/xen_driver.c, src/xenxs/xen_sxpr.c, src/xenxs/xen_xm.c: Update for internal API changes * src/security/security_selinux.c, src/security/virt-aa-helper.c: Only label consoles that aren't a copy of the serial device * src/qemu/qemu_command.c, src/qemu/qemu_driver.c, src/qemu/qemu_process.c, src/uml/uml_conf.c, src/uml/uml_driver.c: Support multiple console devices * tests/qemuxml2xmltest.c, tests/qemuxml2argvtest.c: Extra tests for multiple virtio consoles. Set QEMU_CAPS_CHARDEV for all console /channel tests * tests/qemuxml2argvdata/qemuxml2argv-channel-virtio-auto.args, tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.args tests/qemuxml2argvdata/qemuxml2argv-console-virtio.args: Update for correct chardev syntax * tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.args, tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.xml: New test file 2011-11-03 Daniel P. Berrange Remove translations in socket test case The test case errors should not be translated since they're only targetted at developers, not users. * tests/virnetsockettest.c: Remove error reporting with translations 2011-11-03 Guido Günther virnetsockettest: Use a temporary directory in /tmp to avoid exceeding UNIX_PATH_MAX 2011-11-02 Guido Günther Use ENAMETOOLONG if the the socket path is longer than UNIX_PATH_MAX 2011-11-02 Eric Blake xen: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/xen/xen_hypervisor.c (xenHypervisorGetSchedulerParameters): Allow fewer than max. * src/xen/xend_internal.c (xenDaemonGetSchedulerParameters): Likewise. 2011-11-02 Eric Blake test: drop redundant check libvirt.c guarantees that nparams is non-zero for scheduler parameters. * src/test/test_driver.c (testDomainGetSchedulerParamsFlags): Drop redundant check. Avoid strcpy. 2011-11-02 Eric Blake lxc: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. Also, libvirt.c filters out nparams of 0 for scheduler parameters. * src/lxc/lxc_driver.c (lxcDomainGetMemoryParameters): Allow fewer than max. (lxcGetSchedulerParametersFlags): Drop redundant check. 2011-11-02 Eric Blake libxl: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/libxl/libxl_driver.c (libxlDomainGetSchedulerParametersFlags): Allow fewer than max. 2011-11-02 Eric Blake esx: allow getting < max typed parameters Allow the user to call with nparams too small, per API documentation. * src/esx/esx_driver.c (esxDomainGetMemoryParameters): Drop redundant check. (esxDomainGetSchedulerParametersFlags): Allow fewer than max. 2011-11-02 Eric Blake API: document scheduler parameter names Document the parameter names that will be used by virDomain{Get,Set}SchedulerParameters{,Flags}, rather than hard-coding those names in each driver, to match what is done with memory, blkio, and blockstats parameters. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SCHEDULER_CPU_SHARES) (VIR_DOMAIN_SCHEDULER_VCPU_PERIOD) (VIR_DOMAIN_SCHEDULER_VCPU_QUOTA, VIR_DOMAIN_SCHEDULER_WEIGHT) (VIR_DOMAIN_SCHEDULER_CAP, VIR_DOMAIN_SCHEDULER_RESERVATION) (VIR_DOMAIN_SCHEDULER_LIMIT, VIR_DOMAIN_SCHEDULER_SHARES): New field name macros. * src/qemu/qemu_driver.c (qemuSetSchedulerParametersFlags) (qemuGetSchedulerParametersFlags): Use new defines. * src/test/test_driver.c (testDomainGetSchedulerParamsFlags) (testDomainSetSchedulerParamsFlags): Likewise. * src/xen/xen_hypervisor.c (xenHypervisorGetSchedulerParameters) (xenHypervisorSetSchedulerParameters): Likewise. * src/xen/xend_internal.c (xenDaemonGetSchedulerParameters) (xenDaemonSetSchedulerParameters): Likewise. * src/lxc/lxc_driver.c (lxcSetSchedulerParametersFlags) (lxcGetSchedulerParametersFlags): Likewise. * src/esx/esx_driver.c (esxDomainGetSchedulerParametersFlags) (esxDomainSetSchedulerParametersFlags): Likewise. * src/libxl/libxl_driver.c (libxlDomainGetSchedulerParametersFlags) (libxlDomainSetSchedulerParametersFlags): Likewise. 2011-11-02 Daniel P. Berrange Fix off-by-one printing month in logging code The field 'mon' in 'struct tm' gives months 0-11, where as humans tend to expect months 1-12. Thus the month number needing adjusting by 1 * src/util/logging.c: Use human friendly month number 2011-11-02 Daniel P. Berrange Add missing param initialization in qemuDomainBlockStatsFlags * src/qemu/qemu_driver.c: Fix use of uninitialized 'params' 2011-11-02 Wen Congyang fix crash when starting network commit 27908453 introduces a regression, and it will cause libvirt crashed when starting network. The reason is that tapfd may be NULL, but we dereference it without checking whether it is NULL. 2011-11-01 Eric Blake qemu: allow getting < max typed parameters Since all virTypedParameter APIs allow us to return the number of slots we actually populated, we should allow the user to call with nparams too small (without overrunning their array) or too large (ignoring the tail of the array that we can't fill), rather than requiring that they get things exactly right. Making this change will make it easier for a future patch to introduce VIR_TYPED_PARAM_STRING, with filtering in libvirt.c rather than in every single driver, since users already have to be prepared for *nparams to be smaller on exit than on entry. * src/qemu/qemu_driver.c (qemuDomainGetBlkioParameters) (qemuDomainGetMemoryParameters): Allow variable nparams on entry. (qemuGetSchedulerParametersFlags): Drop redundant check. (qemudDomainBlockStats, qemudDomainBlockStatsFlags): Rename... (qemuDomainBlockStats, qemuDomainBlockStatsFlags): ...to this. Don't return unavailable stats. 2011-11-01 Eric Blake docs: improve typed parameter documentation virDomainBlockStatsFlags was missing a check that was present in virDomainGetMemoryParameters. Additionally, I found that the existing descriptions were a bit hard to read. A later patch will fix qemu to return fewer than max parameters if @nparams was too small on input. * src/libvirt.c (virDomainGetMemoryParameters) (virDomainGetBlkioParameters, virDomainGetSchedulerParameters) (virDomainGetSchedulerParametersFlags): Tweak documentation wording. (virDomainBlockStatsFlags): Likewise, and add sanity check. 2011-11-01 Daniel P. Berrange Don't overwrite error message during VM cleanup If an LXC VM fails to start, quite a few cleanup paths will result in the original error message being overwritten. Some other cleanup paths also forgot to actually terminate the VM. * src/lxc/lxc_driver.c: Ensure VM is terminated on startup failure and preserve original error 2011-11-01 Daniel P. Berrange Add support for probing filesystem with libblkid The LXC code for mounting container filesystems from block devices tries all filesystems in /etc/filesystems and possibly those in /proc/filesystems. The regular mount binary, however, first tries using libblkid to detect the format. Add support for doing the same in libvirt, since Fedora's /etc/filesystems is missing many formats, most notably ext4 which is the default filesystem Fedora uses! * src/Makefile.am: Link libvirt_lxc to libblkid * src/lxc/lxc_container.c: Probe filesystem format with libblkid 2011-11-01 Daniel P. Berrange Fix error message when failing to detect filesystem If we looped through /etc/filesystems trying to mount with each type and failed all options, we forget to actually raise an error message. * src/lxc/lxc_container.c: Raise error if unable to detect the filesystems. Also fix existing error message 2011-11-01 Daniel P. Berrange Workaround for broken kernel autofs mounts The kernel automounter is mostly broken wrt to containers. Most notably if you start a new filesystem namespace and then attempt to unmount any autofs filesystem, it will typically fail with a weird error message like Failed to unmount '/.oldroot/sys/kernel/security':Too many levels of symbolic links Attempting to detach the autofs mount using umount2(MNT_DETACH) will also fail with the same error. Therefore if we get any error on unmount()ing a filesystem from the old root FS when starting a container, we must immediately break out and detach the entire old root filesystem (ignoring any mounts below it). This has the effect of making the old root filesystem inaccessible to anything inside the container, but at the cost that the mounts live on in the kernel until the container exits. Given that SystemD uses autofs by default, we need LXC to be robust this scenario and thus this tradeoff is worthwhile. * src/lxc/lxc_container.c: Detach root filesystem if any umount operation fails. 2011-11-01 Daniel P. Berrange Correctly handle '*' in /etc/filesystems The /etc/filesystems file can contain a '*' on the last line to indicate that /proc/filessystems should be tried next. We have a check that this '*' only occurs on the last line. Unfortunately when we then start reading /proc/filesystems, we mistakenly think we've seen '*' in /proc/filesystems and fail * src/lxc/lxc_container.c: Skip '*' validation when we're reading /proc/filesystems 2011-11-01 Daniel P. Berrange Ensure errno is valid when returning from lxcContainerWaitForContinue Only some of the return paths of lxcContainerWaitForContinue will have set errno. In other paths we need to set it manually to avoid the caller getting a random stale errno value * src/lxc/lxc_container.c: Set errno in lxcContainerWaitForContinue 2011-11-01 Daniel P. Berrange Create /var/lib/libvirt/filesystems for LXC trees We already have a /var/lib/libvirt/images for OS install images. We need a separate /var/lib/libvirt/filesystems for OS install trees, since SELinux labelling will be different * libvirt.spec.in: Add /var/lib/libvirt/filesystems * src/Makefile.am: Create /var/lib/libvirt/filesystems 2011-11-01 Matthias Bolte esx: Support folders in the path of vpx:// connection URIs Allow the datacenter and compute resource parts of the path to be prefixed with folders. Therefore, the way the path is parsed has changed. Before, it was split in 2 or 3 items and the items' meanings were determined by their positions. Now the path can have 2 or more items and the the vCenter server is asked whether a folder, datacenter of compute resource with the specified name exists at the current hierarchy level. Before the datacenter and compute resource lookup automatically traversed folders during lookup. This is logic got removed and folders have to be specified explicitly. The proper datacenter path including folders is now used when accessing a datastore over HTTPS. This makes virsh dumpxml and define work for datacenters in folders. https://bugzilla.redhat.com/show_bug.cgi?id=732676 2011-11-01 Patrice LACHANCE esx: Support vSphere 5.x And virtual hardware version 8. 2011-11-01 Wen Ruo Lv Fix URI alias prefix matching with /etc/libvirt/libvirt.conf below: uri_aliases = [ "hail=qemu:///system", "sleet=qemu+ssh://root 9 115 122 57/system", "sam=qemu+unix:///system?socket=/var/run/libvirt/libvirt-sock", ] Neither "virsh -c hailly" nor "hai" should result in matching "hail=qemu:///system" Fix URI alias prefix matching when connecting 2011-11-01 Eric Blake docs: fix typo in / example * docs/formatdomain.html.in: Use dev, not def. Reported by Alexander Biryukov. 2011-11-01 Michal Privoznik ServerClient: Flush cached data If daemon is using SASL it reads client data into a cache. This cache is big (usually 65KB) and can thus contain 2 or more messages. However, on socket event we can dispatch only one message. So if we read two messages at once, the second will not be dispatched as the socket event goes away with filling the cache. Moreover, when dispatching the cache we need to remember to take care of client max requests limit. 2011-11-01 Daniel P. Berrange Fix storage pool source comparison to avoid comparing with self If we are comparing storage pools we must skip comparing with ourself, so that re-defining an existing pool works * conf/storage_conf.c: Skip self when comparing 2011-10-31 Sage Weil qemu: pass virConnectPtr into Domain{Attach,Detach}* The qemu RBD driver needs access to the conn in order to get the secret needed for connecting to the ceph cluster. 2011-10-31 Alex Jia qemu: plug memory leak Detected by Coverity. Leak introduced in commit 6cabc0b. * src/qemu/qemu_command.c: Clean up on failure. 2011-10-31 Matthias Bolte python: Fix documentation of virStream recv This was fixed in be757a3f7baf93b for libvirt.c. 2011-10-31 Michal Privoznik startupPolicty: Minor cleanups This patch does some cleanups to my previous startupPolicy patchset. 2011-10-30 Osier Yang qemu: Restore the original states of PCI device when restarting daemon To support "managed" mode of host PCI device, we record the original states (unbind_from_stub, remove_slot, and reprobe) so that could reattach the device to host with original driver. But there is no XML for theses attrs, and thus after daemon is restarted, we lose the original states. It's easy to reproduce: 1) virsh start domain 2) virsh attach-device dom hostpci.xml (in 'managed' mode) 3) service libvirtd restart 4) virsh destroy domain You will see the device won't be bound to the original driver if there was one. This patch is to solve the problem by introducing internal XML (won't be dumped to user, only dumped to status XML). The XML is: Which will be child node of .... (only for PCI device). A new struct "virDomainHostdevOrigStates" is introduced for the XML, and the according members are updated when preparing the PCI device. And function "qemuUpdateActivePciHostdevs" is modified to honor the original states. Use of qemuGetPciHostDeviceList is removed in function "qemuUpdateActivePciHostdevs", and the "managed" value of the device config is honored by the change. This fixes another problem alongside: qemuGetPciHostDeviceList set the device as "managed" force regardless of whether the device is configured as "managed='yes'" or not in XML, which is not right. 2011-10-29 Matthias Bolte vbox: Add support for VirtualBox 4.1 Deal with the incompatible changes in the VirtualBox 4.1 API. INetworkAdapter has its different AttachTo* method replaced by a settable attachmentType property. The maximum number of network adapters is now requestable per chipset type. The OpenMedium method got a bool parameter to request opening a medium under a new IID. 2011-10-29 Matthias Bolte vbox: Support shared folders Shared folders are handled as filesystems and can also be hotplugged. 2011-10-29 Matthias Bolte xenapi: Improve error reporting in xenapiOpen once again privP->session->error_description is a list and in order to get the complete error message all parts of the list should be concatenated. xenapiSessionErrorHandler does this when its third parameter is NULL. The current code discards all but the first part of the error message resulting in a potentially incomplete error message. This partly reverts 006be75ee214f9b4, that tried to avoid reporting a (null) in the error message. The actual problem is more general in returnErrorFromSession that might return NULL if there is no error. Make sure that returnErrorFromSession return non-NULL always. Also don't skip the last error message part. 2011-10-29 Roopa Prabhu macvtap: Fix error return value convention/inconsistencies - changed some return 1's to return -1 - changed if (rc) error checks to if (rc < 0) - fixed some other minor convention violations I might have missed some. Can fix in another patch or can respin Reported-by: Eric Blake Reported-by: Laine Stump 2011-10-29 Josh Durgin Use a common xml type for ceph secret usage. The types used in domaincommon.rng and secret.rng should be the same. Move genericName to basictypes.rng, then drop redundant types now that secret.rng uses basictypes.rng. 2011-10-28 Xu He Jie pci address conflict when virtio disk with drive type When using the xml as below: ------------------------------------------------------ /home/soulxu/data/work-code/qemu-kvm/x86_64-softmmu/qemu-system-x86_64
------------------------------------------------------ Then can't startup qemu, the error message as below: virsh # start test-vm error: Failed to start domain test-vm error: internal error process exited while connecting to monitor: qemu-system-x86_64: -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3: PCI: slot 3 function 0 not available for virtio-balloon-pci, in use by virtio-blk-pci qemu-system-x86_64: -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3: Device 'virtio-balloon-pci' could not be initialized So adding check for bus type and address type. Only the address of pci type support by virtio bus. 2011-10-28 Eric Blake secret: fix bad patch application In hand-applying Josh and Sage's patch, I missed out on a break. * src/conf/secret_conf.c (virSecretDefFree): Fix my botch. 2011-10-28 Josh Durgin storage: add auth to virDomainDiskDef Add additional fields to let you specify the how to authenticate with a disk. The secret to use may be referenced by a usage string or a UUID, i.e.: or 2011-10-28 Sage Weil secret: add Ceph secret type Add a new secret type to store a Ceph authentication key. The name is simply an identifier for easy human reference. The xml looks like this: 0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f mycluster_admin 2011-10-28 Alex Jia qemu: plug memory leak Leak introduced in commit c1bc3d89. Detected by valgrind: ==18462== 1,100 bytes in 1 blocks are definitely lost in loss record 183 of 184 ==18462== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==18462== by 0x4A06167: realloc (vg_replace_malloc.c:525) ==18462== by 0x4AADBB: virReallocN (memory.c:161) ==18462== by 0x4A975E: virBufferGrow (buf.c:117) ==18462== by 0x4A9D92: virBufferVasprintf (buf.c:290) ==18462== by 0x4A9EF7: virBufferAsprintf (buf.c:263) ==18462== by 0x429488: qemuBuildControllerDevStr (qemu_command.c:1993) ==18462== by 0x42C4B6: qemuBuildCommandLine (qemu_command.c:3803) ==18462== by 0x41A604: testCompareXMLToArgvHelper (qemuxml2argvtest.c:124) ==18462== by 0x41BB81: virtTestRun (testutils.c:141) ==18462== by 0x416DFF: mymain (qemuxml2argvtest.c:369) ==18462== by 0x41B277: virtTestMain (testutils.c:696) ==18462== ==18462== LEAK SUMMARY: ==18462== definitely lost: 1,100 bytes in 1 blocks ==18462== indirectly lost: 0 bytes in 0 blocks * src/qemu/qemu_command.c (qemuBuildCommandLine): Clean up on success. 2011-10-28 Eric Blake qemu: avoid leaking uninit data from hotplug to dumpxml Detected by Coverity. The fix in 2c27dfa didn't catch all bad instances of memcpy(). Thankfully, on further analysis, all of the problematic uses are only triggered by old qemu that lacks -device. * src/qemu/qemu_hotplug.c (qemuDomainAttachPciDiskDevice) (qemuDomainAttachNetDevice, qemuDomainAttachHostPciDevice): Init all fields since monitor only populates some of them. 2011-10-28 Ryota Ozaki util: Fix virUUIDGeneratePseudoRandomBytes It forgets to move a pointer to a buffer for UUID and as a result fills only the first byte of the buffer. 2011-10-28 Daniel P. Berrange Implement RPC driver support for virDomainOpenGraphics Since it needs to access file descriptors passed in the msg, the RPC driver for virDomainOpenGraphics needs to be manually implemented. * daemon/remote.c: RPC server dispatcher * src/remote/remote_driver.c: RPC client dispatcher * src/remote/remote_protocol.x: Define protocol 2011-10-28 Daniel P. Berrange Extend RPC server to allow FD passing The RPC server classes are extended to allow FDs to be received from clients with calls. There is not currently any way for a procedure to pass FDs back to the client with replies * daemon/remote.c, src/rpc/gendispatch.pl: Change virNetMessageHeaderPtr param to virNetMessagePtr in dispatcher impls * src/rpc/virnetserver.c, src/rpc/virnetserverclient.c, src/rpc/virnetserverprogram.c, src/rpc/virnetserverprogram.h: Extend to support FD passing 2011-10-28 Daniel P. Berrange Add client side support for FD passing Extend the RPC client code to allow file descriptors to be sent to the server with calls, and received back with replies. * src/remote/remote_driver.c: Stub extra args * src/libvirt_private.syms, src/rpc/virnetclient.c, src/rpc/virnetclient.h, src/rpc/virnetclientprogram.c, src/rpc/virnetclientprogram.h: Extend APIs to allow FD passing 2011-10-28 Daniel P. Berrange Extend RPC protocol to allow FD passing Define two new RPC message types VIR_NET_CALL_WITH_FDS and VIR_NET_REPLY_WITH_FDS. These message types are equivalent to VIR_NET_CALL and VIR_NET_REPLY, except that between the message header, and payload there is a 32-bit integer field specifying how many file descriptors have been passed. The actual file descriptors are sent/recv'd out of band. * src/rpc/virnetmessage.c, src/rpc/virnetmessage.h, src/libvirt_private.syms: Add support for handling passed file descriptors * src/rpc/virnetprotocol.x: Extend protocol for FD passing 2011-10-28 Daniel P. Berrange Add APIs for virNetSocket for sending/receiving file descriptors Add APIs to the virNetSocket object, to allow file descriptors to be sent/received over UNIX domain socket connections * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h, src/libvirt_private.syms: Add APIs for FD send/recv 2011-10-28 Daniel P. Berrange Wire up QEMU implementation for virDomainOpenGraphics The QEMU monitor command 'add_client' can be used to connect to a VNC or SPICE graphics display. This allows for implementation of the virDomainOpenGraphics API * src/qemu/qemu_driver.c: Implement virDomainOpenGraphics * src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h, src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h, src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add binding for 'add_client' command 2011-10-28 Daniel P. Berrange Extend graphics event to include UNIX socket Not all VNC/SPICE servers use a TCP socket for their connections. It is possible to configure a UNIX socket server. The graphics event must thus include a UNIX socket address type. * include/libvirt/libvirt.h.in: Add UNIX socket address type for graphics event * src/qemu/qemu_monitor_json.c: Add 'unix' string to address type enum 2011-10-28 Daniel P. Berrange Introduce the virDomainOpenGraphics API The virDomainOpenGraphics API allows a libvirt client to pass in a file descriptor for an open socket pair, and get it connected to the graphics display of the guest. This is limited to working with local libvirt hypervisors connected over a UNIX domain socket, since it will use UNIX FD passing * include/libvirt/libvirt.h.in: Define virDomainOpenGraphics * src/driver.h: Define driver for virDomainOpenGraphics * src/libvirt_public.syms, src/libvirt.c: Entry point for virDomainOpenGraphics * src/libvirt_internal.h: VIR_DRV_FEATURE_FD_PASSING 2011-10-28 Daniel P. Berrange Remove trailing whitespace from all xfig files The 5th line of every xfig file has 'Letter ' which annoys GITs trailing-whitespace commit hook. Hand edit the files to remove that trailing whitespace * docs/*.fig: Remove trailing whitespace 2011-10-28 Daniel P. Berrange Add documentation about migration. This adds a page documenting many aspects of migration: - The types of migration (managed direct, p2p, unmanaged direct) - Data transports (native, tunnelled) - Migration URIs - Config file handling - Example scenarios * libvirt.css: Rules for data tables and diagrams * Makefile.am: Include extra png/fig files * migration-managed-direct.fig, migration-managed-direct.png, migration-managed-direct.png, migration-managed-p2p.png, migration-native.fig, migration-native.png, migration-tunnel.fig, migration-tunnel.png, migration-unmanaged-direct.fig, migration-unmanaged-direct.png: Diagrams of migration * migration.html.in, sitemap.html.in: New migration doc 2011-10-28 Ryota Ozaki virsh: Fix error message on vol-create-from failure If vol-create-from is failed due to 'input volume not found', virsh outputs like this: $ sudo virsh vol-create-from testpool test-vol.xml test.img error: failed to get vol 'test.img', specifying --pool might help error: Storage volume not found: no storage vol with matching path However, '--pool' is incorrect because it is already specified as second argument ('testpool' in this case). It should be "--inputpool". The patch fixes this by using pooloptname, which will be "inputpool" in this case and "pool" in other cases, as error message. 2011-10-28 Tyler Coumbes bridge: modify for use when sVirt is enabled with qemu This refactors the TAP creation code out of brAddTap into a new function brCreateTap to allow it to be used on its own. I have also changed ifSetInterfaceMac to brSetInterfaceMac and exported it since it is will be needed by code outside of util/bridge.c in the next patch. AUTHORS | 1 + src/libvirt_bridge.syms | 2 + src/util/bridge.c | 116 +++++++++++++++++++++++++++++++---------------- src/util/bridge.h | 9 ++++ 4 files changed, 89 insertions(+), 39 deletions(-) 2011-10-27 Jiri Denemark Use virXMLSaveFile when writing XML config Introduce virXMLSaveFile as a wrapper for virFileRewrite Every time we write XML into a file we call virEmitXMLWarning to write a warning that the file is automatically generated. virXMLSaveFile simplifies this into a single step and makes rewriting existing XML file safe by using virFileRewrite internally. Introduce virFileRewrite for safe file rewrite When saving config files we just overwrite old content of the file. In case something fails during that process (e.g. disk gets full) we lose both old and new content. This patch makes the process more robust by writing the new content into a separate file and only if that succeeds the original file is atomically replaced with the new one. 2011-10-27 Daniel P. Berrange Add a systemtap script for watching QEMU monitor interactions This change adds some systemtap/dtrace probes to the QEMU monitor client code. In particular it allows watching of all operations for a VM * examples/systemtap/qemu-monitor.stp: Watch all monitor commands * src/Makefile.am: Passing libdir/bindir/sbindir to dtrace2systemtap.pl * src/dtrace2systemtap.pl: Accept libdir/bindir/sbindir as args and look for '# binary:' comment to mark probes against libvirtd vs libvirt.so * src/qemu/qemu_monitor.c, src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_text.c: Add probes for key functions 2011-10-27 Peter Krempa lxc: Revert zeroing count of allocated items if VIR_REALLOC_N fails Previous commit clears number of items alocated in lxcSetupLoopDevices if VIR_REALLOC_N fails. In that case, the pointer is not NULL, and causes leaking FDs that have been allocated. * src/lxc/lxc_controller.c: revert zeroing array size 2011-10-27 Daniel P. Berrange Fix typo in virFileAccessibleAs * src/util/util.c: s/git_t/gid_t/ in parameter list of virFileAccessibleAs 2011-10-27 Alex Jia lxc: avoid null deref on lxcSetupLoopDevices failure If the function lxcSetupLoopDevices(def, &nloopDevs, &loopDevs) failed, the variable loopDevs will keep a initial NULL value, however, the function VIR_FORCE_CLOSE(loopDevs[i]) will directly deref it. This patch also fixes returning a bogous number of devices from lxcSetupLoopDevices on an error path. * rc/lxc/lxc_controller.c: fixed a null pointer dereference. 2011-10-27 Alex Jia lxc: avoid missing '{' in the function Cppcheck detected a syntaxError on lxcDomainInterfaceStats. * src/lxc/lxc_driver.c: fixed missing '{' in the function lxcDomainInterfaceStats. 2011-10-26 Eric Blake qemu: simplify use of HAVE_YAJL Rather than making all clients of monitor commands that are JSON-only check whether yajl support was compiled in, it is simpler to just avoid setting the capability bit up front if we can't use the capability. * src/qemu/qemu_capabilities.c (qemuCapsComputeCmdFlags): Only set capability bit if we also have yajl library to use it. * src/qemu/qemu_driver.c (qemuDomainReboot): Drop #ifdefs. * src/qemu/qemu_process.c (qemuProcessStart): Likewise. * tests/qemuhelptest.c (testHelpStrParsing): Pass test even without yajl. * tests/qemuxml2argvtest.c (mymain): Simplify use of json flag. * tests/qemuxml2argvdata/qemuxml2argv-disk-drive-error-*.args: Update expected results to match. 2011-10-26 Eric Blake snapshot: minor cleanups from reviewing indentation Break some long lines, and use more efficient functions when possible, such as relying on virBufferEscapeString to skip output on a NULL arg. Ensure that output does not embed newlines, since auto-indent won't work in those situations. * src/conf/domain_conf.c (virDomainTimerDefFormat): Break output lines. (virDomainDefFormatInternal, virDomainDiskDefFormat) (virDomainActualNetDefFormat, virDomainNetDefFormat) (virDomainHostdevDefFormat): Minor cleanups. 2011-10-26 Eric Blake snapshot: simplify indentation of disk encryption xml Use auto-indent in more places. * src/conf/storage_encryption_conf.h (virStorageEncryptionFormat): Drop parameter. * src/conf/storage_encryption_conf.c (virStorageEncryptionFormat) (virStorageEncryptionSecretFormat): Simplify with auto-indent. * src/conf/domain_conf.c (virDomainDiskDefFormat): Adjust caller. * src/conf/storage_conf.c (virStorageVolTargetDefFormat): Likewise. 2011-10-26 Eric Blake snapshot: simplify indentation of nwfilter Fixing this involved some refactoring of common code out of domain_conf and nwfilter_conf into nwfilter_params. * src/conf/nwfilter_params.h (virNWFilterFormatParamAttributes): Adjust signature. * src/conf/nwfilter_params.c (_formatParameterAttrs) (virNWFilterFormatParamAttributes): Adjust indentation handling, and handle filterref here. (formatterParam): Delete unused struct. * src/conf/domain_conf.c (virDomainNetDefFormat): Adjust caller. * src/conf/nwfilter_conf.c (virNWFilterIncludeDefFormat): Likewise. 2011-10-26 Eric Blake storage: avoid null deref on qemu-img failure Detected by Coverity. Only possible if qemu-img gives bogus output, but we might as well be robust. * src/storage/storage_backend.c (virStorageBackendQEMUImgBackingFormat): Check for strstr failure. 2011-10-26 Eric Blake build: avoid RHEL 5 build failure on LXC Per the discussion here, LXC on RHEL 5 makes no sense. https://www.redhat.com/archives/libvir-list/2011-September/msg01169.html * configure.ac (with_lxc): Reject RHEL 5.x LXC as too old. 2011-10-25 Eric Blake build: use gnulib fdatasync Commit 1726a73 hacked around MacOS' lack of fdatasync, since gnulib did not have it at the time. But now that we use newer gnulib, we can avoid the hack. * bootstrap.conf (gnulib_modules): Add fdatasync. * configure.ac (AC_CHECK_FUNCS_ONCE): Drop our own check. 2011-10-25 Daniel P. Berrange Add REMOTE_PROC_DOMAIN_EVENT_DISK_CHANGE to remote_protocol-structs * src/remote_protocol-structs: Add new event 2011-10-25 Stefan Berger nwfilter: extend schema to support new targets Extend the nwfilter schema to support the continue and return targets. 2011-10-25 Michal Privoznik util: Add virFileAccessibleAs to private symbols Commit 458b7099b2e791b58f1286002974927d3e8bcc80 introduced this function. However it was not added into libvirt_private.syms so the compilation may not succeed on some hosts. startupPolicy: Emit event on disk source dropping If a disk source gets dropped because it is not accessible, mgmt application might want to be informed about this. Therefore we need to emit an event. The event presented in this patch is however a bit superset of what written above. The reason is simple: an intention to be easily expanded, e.g. on 'user ejected disk in guest' events. Therefore, callback gets source string and disk alias (which should be unique among a domain) and reason (an integer); qemu: implement startupPolicy This patch implements on_missing feature in qemu driver. Upon qemu startup process an accessibility of CDROMs and floppy disks is checked. The source might get dropped if unavailable and on_missing is set accordingly. No event is emit thought. Look for follow up patch. qemu: Move device alias assigning before command line construction This patch is rather cosmetic as it only moves device alias assignation from command line construction just before that. However, it is needed in connotation of previous and next patch. util: Create virFileAccessibleAs function This function checks if a given path is accessible under given uid and gid. 2011-10-25 Michal Privoznik conf: Introduce optional startupPolicy attribute for cdrom and floppy This attribute says what to do with cdrom (or floppy) if the source is missing. It accepts: - mandatory - fail if missing for any reason (the default) - requisite - fail if missing on boot up, drop if missing on migrate/restore/revert - optional - drop if missing at any start attempt. However, this patch introduces only XML part of this new functionality. 2011-10-25 Eric Blake storage: make previous leak less likely to regress Splitting into two functions allows the user to call the right function, rather than having to remember that a *Free function is an exception to the rule. * src/conf/storage_conf.h (virStoragePoolSourceClear): New function. * src/libvirt_private.syms (storage_conf.h): Export it. * src/conf/storage_conf.c (virStoragePoolSourceFree): Split... (virStoragePoolSourceClear): ...into new function. (virStoragePoolDefFree, virStoragePoolDefParseSourceString): Update callers. * src/test/test_driver.c (testStorageFindPoolSources): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendFileSystemNetFindPoolSourcesFunc) (virStorageBackendFileSystemNetFindPoolSources): Likewise. * src/storage/storage_backend_iscsi.c (virStorageBackendISCSIFindPoolSources): Likewise. * src/storage/storage_backend_logical.c (virStorageBackendLogicalFindPoolSources): Likewise. 2011-10-25 Eric Blake storage: plug iscsi memory leak Detected by Coverity. virStoragePoolSourceFree does not free the actual passed-in pointer. A bigger patch would be to rename it virStoragePoolSourceClear to match behavior, or even split it into two functions depending on needed behavior; but this is the minimal fix to the one location out of eight that leaked memory. * src/storage/storage_backend_iscsi.c (virStorageBackendISCSIFindPoolSources): Free memory. 2011-10-24 Eric Blake waitpid: improve safety Based on a report by Coverity. waitpid() can leak resources if it fails with EINTR, so it should never be used without checking return status. But we already have a helper function that does that, so use it in more places. * src/lxc/lxc_container.c (lxcContainerAvailable): Use safer virWaitPid. * daemon/libvirtd.c (daemonForkIntoBackground): Likewise. * tests/testutils.c (virtTestCaptureProgramOutput, virtTestMain): Likewise. * src/libvirt.c (virConnectAuthGainPolkit): Simplify with virCommand. 2011-10-24 Eric Blake qemu: avoid leaking uninit data from hotplug to dumpxml Detected by Coverity. Both text and JSON monitors set only the bus and unit fields, which means driveAddr.controller spends life as garbage on the stack, and is then memcpy()'d into the in-memory representation which the user can see via dumpxml. * src/qemu/qemu_hotplug.c (qemuDomainAttachSCSIDisk): Only copy defined fields. 2011-10-24 Ryota Ozaki virsh: Fix vol-info's 'Type' output We have a new vol type "dir" in addition to "file" and "block", but virsh doesn't know it. Fix it. Additionally, the patch lets virsh output "unknown" if not matched any of them. 2011-10-21 Ryota Ozaki docs: Document filesystem type='block' for LXC Commit 77791dc0e allows LXC to use a host block device as a guest filesystem, but it isn't documented yet. 2011-10-21 David L Stevens support continue/return targets in nwfilter This patch adds support for "continue" and "return" actions in filter rules. 2011-10-21 Eric Blake snapshot: simplify indentation of network xml More simplifications possible due to auto-indent. Also, within was only using 6 instead of 8 spaces. * src/util/network.h (virVirtualPortProfileFormat) (virBandwidthDefFormat): Alter signature. * src/util/network.c (virVirtualPortProfileFormat) (virBandwidthDefFormat): Alter indentation. (virBandwidthChildDefFormat): Tweak to make use easier. * src/conf/network_conf.c (virPortGroupDefFormat) (virNetworkDefFormat): Adjust callers. * src/conf/domain_conf.c (virDomainNetDefFormat): Likewise. (virDomainActualNetDefFormat): Likewise, and fix bandwidth indentation. 2011-10-21 Eric Blake snapshot: simplify indentation of cpu features Auto-indent makes life a bit easier; this patch also drops unused arguments and replaces a misspelled flag name with two entry points instead, so that callers don't have to worry about how much spacing is present when embedding cpu elements. * src/conf/cpu_conf.h (virCPUFormatFlags): Delete. (virCPUDefFormat): Drop unused argument. (virCPUDefFormatBuf): Alter signature. (virCPUDefFormatBufFull): New prototype. * src/conf/cpu_conf.c (virCPUDefFormatBuf): Split... (virCPUDefFormatBufFull): ...into new function. (virCPUDefFormat): Adjust caller. * src/conf/domain_conf.c (virDomainDefFormatInternal): Likewise. * src/conf/capabilities.c (virCapabilitiesFormatXML): Likewise. * src/cpu/cpu.c (cpuBaselineXML): Likewise. * tests/cputest.c (cpuTestCompareXML): Likewise. 2011-10-21 Eric Blake snapshot: simplify indentation of sysinfo The improvements to virBuffer, along with a paradigm shift to pass the original buffer through rather than creating a second buffer, allow us to shave off quite a few lines of code. * src/util/sysinfo.h (virSysinfoFormat): Alter signature. * src/util/sysinfo.c (virSysinfoFormat, virSysinfoBIOSFormat) (virSysinfoSystemFormat, virSysinfoProcessorFormat) (virSysinfoMemoryFormat): Change indentation parameter. * src/conf/domain_conf.c (virDomainSysinfoDefFormat): Adjust caller. * src/qemu/qemu_driver.c (qemuGetSysinfo): Likewise. 2011-10-21 Eric Blake snapshot: test domainsnapshot indentation Add a test for the simple parts of my indentation changes, and fix the fallout. * tests/domainsnapshotxml2xmltest.c: New test. * tests/Makefile.am (domainsnapshotxml2xmltest_SOURCES): Build it. * src/conf/domain_conf.c (virDomainSnapshotDefFormat): Avoid NULL deref, match documented order. * src/conf/domain_conf.h (virDomainSnapshotDefFormat): Add const. * tests/domainsnapshotxml2xmlout/all_parameters.xml: Tweak output. * tests/domainsnapshotxml2xmlout/disk_snapshot.xml: Likewise. * tests/domainsnapshotxml2xmlout/full_domain.xml: Likewise. * .gitignore: Exempt new binary. 2011-10-21 Eric Blake snapshot: indent domain xml when nesting is the first public instance of being used as a sub-element, although we have two other private uses (runtime state, and migration cookie). Although indentation has no effect on XML parsing, using it makes the output more consistent. This uses virBuffer auto-indentation to obtain the effect, for all but the portions of that are not generated a line at a time into the same virBuffer. Further patches will clean up the remaining problems. * src/conf/domain_conf.h (virDomainDefFormatInternal): New prototype. * src/conf/domain_conf.c (virDomainDefFormatInternal): Export. (virDomainObjFormat, virDomainSnapshotDefFormat): Update callers. * src/libvirt_private.syms (domain_conf.h): Add new export. * src/qemu/qemu_migration.c (qemuMigrationCookieXMLFormat): Use new function. (qemuMigrationCookieXMLFormatStr): Update caller. 2011-10-21 Eric Blake virbuf: add auto-indentation support Rather than having to adjust all callers in a chain to deal with indentation, it is nicer to have virBuffer do auto-indentation. * src/util/buf.h (_virBuffer): Increase size. (virBufferAdjustIndent, virBufferGetIndent): New prototypes. * src/libvirt_private.syms (buf.h): Export new functions. * src/util/buf.c (virBufferAdjustIndent, virBufferGetIndent): New functions. (virBufferSetError, virBufferAdd, virBufferAddChar) (virBufferVasprintf, virBufferStrcat, virBufferURIEncodeString): Implement auto-indentation. * tests/virbuftest.c (testBufAutoIndent): Test it. (testBufInfiniteLoop): Don't rely on internals. Idea by Daniel P. Berrange. 2011-10-21 Eric Blake virbuf: more detailed error reporting The next patch wants to add some sanity checking, which would be a different error than ENOMEM. Many existing callers blindly report OOM failure if virBuf reports an error, and this will be wrong in the (unlikely) case that they actually had a usage error instead; but since the most common error really is ENOMEM, I'm not going to fix all callers. Meanwhile, new discriminating callers can react differently depending on what failure happened. * src/util/buf.c (virBufferSetError): Add parameter. (virBufferGrow, virBufferVasprintf, virBufferEscapeString) (virBufferEscapeSexpr, virBufferEscapeShell): Adjust callers. 2011-10-21 Eric Blake virbuf: improve testsuite reporting I had some temporary test failures while working on virbuf improvements in later patches, with output that looked like: Expected [<] Actual [ <] which is pretty hard to figure out. Adding an Offset designation made it much easier to find which particular '<' was at the wrong indentation, to fix the right part of the code. * tests/testutils.c (virtTestDifference): Make it easier to diagnose test failures. 2011-10-21 Eric Blake virbuf: fix const-correctness Although the compiler wasn't complaining (since it was the pointer, rather than what was being pointed to, that was actually const), it looks quite suspicious to call a function with an argument labeled const when the nature of the pointer (virBufferPtr) is hidden behind a typedef. Dropping const makes the function declarations easier to read. * src/util/buf.h: Drop const from all functions that modify buffer argument. * src/util/buf.c (virBufferSetError, virBufferAdd) (virBufferContentAndReset, virBufferFreeAndReset) (virBufferAsprintf, virBufferVasprintf, virBufferEscapeString) (virBufferEscapeSexpr, virBufferEscape): Fix fallout. 2011-10-20 Laine Stump docs: fix incorrect info about routed networks In a recent expansion of the documentation on network forward modes, I incorrectly stated that incoming sessions to guests on routed networks were blocked. This is true for guests on NATed networks, but not routed. This patch corrects that error, and adds a pointer to the nwfilter page for those who do want to restrict incoming sessions to hosts on routed networks. 2011-10-20 tangchen qemu: allow json in domxml-to-native There is a little difference between the output of domxml-to-native and the actual commandline. No matter qemu is in control or readline mode, domxml-to-native always converts it to readline mode. That is because the parameter "monitor_json" for qemuBuildCommandLine() is always set to false in qemuDomainXMLToNative(). 2011-10-20 Eric Blake docs: document managed=yes of hostdev passthrough Clarify some of the effects of managed passthrough devices; with recent changes (commit d093547), a nodedev-reattach is only needed to pair up to an explicit nodedev-dettach (but beware that older virt-manager has a bug where it uses explicit nodedev-dettach under the hood when using the gui to hotplug a hostdev device). * docs/formatdomain.html.in: Mention reattach. * tools/virsh.pod (nodedev): Mention managed mode. 2011-10-20 Daniel P. Berrange Fix virFileOpenTty definition on Win32 Stub out a complete impl of virFileOpenTty to avoid unused parameter warnings * src/util/util.c: Fix virFileOpenTty on Win32 2011-10-20 Hu Tao support setting bandwidth from virsh attach-interface Adds two options, inbound and outbound, to attach-interface to set bandwidth when attaching interfaces 2011-10-19 Serge E. Hallyn lxc: use hand-rolled code in place of unlockpt and grantpt The glibc ones (intentionally) cannot handle ptys opened in a devpts not mounted at /dev/pts. Drop the (un-exported, unused) virFileOpenTtyAt. 2011-10-19 Eric Blake build: ignore test executable * .gitignore: Add exemption. 2011-10-19 Philipp Hahn qemu: Test name-space handling Add test cases for parsing the qemu-name-space. This is based on qemuxml2argv{test,data/}, but can not reside in qemuxml2argv{test,data/} because ... 1. qemuxmlns-qemu-ns-domain.xml is not schema-valid and breaks domainschematest. The test is still important to detect xmlns:qemu bindings to a name-space other than http://libvirt.org/schemas/domain/qemu/1.0 2. they break qemuxml2xml, because the xmlns:qemu binding is moved to the top-level element when converting from argv to xml. 2011-10-19 Philipp Hahn qemu: Fix name-space handling The XML parser for the qemu specific extensions expects the qemu name-space to be bound to the 'qemu' prefix. This is too strict, since the name of the name-space-prefix is only meant as an internal lookup key. Only the associated URI is relevant. ... ... ... ... ... Remove the test for checking the name-space binding on the top-level element. Registering the name-space with XPath is enough. 2011-10-19 Xu He Jie compile: fix undefined reference to gnutls_x509_crt_get_dn with gcc-4.6.1 When I compile libvirt with gcc-4.6.1 in ubuntu 11.10, got error as below: CCLD libvirtd /usr/bin/ld: ../src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_migration.o): undefined reference to symbol 'gnutls_x509_crt_get_dn@@GNUTLS_1_4' /usr/bin/ld: note: 'gnutls_x509_crt_get_dn@@GNUTLS_1_4' is defined in DSO /usr/lib/x86_64-linux-gnu/libgnutls.so so try adding it to the linker command line /usr/lib/x86_64-linux-gnu/libgnutls.so: could not read symbols: Invalid operation collect2: ld returned 1 exit status make[3]: *** [libvirtd] Error 1 It can compile with gcc-4.5.2 in ubuntu 11.04, but it can not compile with gcc-4.6.1 in ubuntu 11.10. I didn't find reason. Does Anyone know the reason or the different between gcc-4.5.2 and gcc-4.6.1? I still provide a patch for this. Just make it is working now. 2011-10-19 Daniel P. Berrange Replace virBufferAdd with virBufferAddLit for const string The statement virBufferAdd(buf, "''", 2); triggers a syntax-check warning * src/util/buf.c: Replace virBufferAdd with virBufferAddLit 2011-10-19 Philipp Hahn documentation: trivial spelling fix 2011-10-19 Daniel P. Berrange Allow for URI aliases when connecting to libvirt This adds support for a libvirt client configuration file either /etc/libvirt/libvirt.conf for privileged clients, or $HOME/.libvirt/libvirt.conf for unprivileged clients. It allows one parameter uri_aliases = [ "hail=qemu+ssh://root@hail.cloud.example.com/system", "sleet=qemu+ssh://root@sleet.cloud.example.com/system", ] Any call to virConnectOpen with a non-NULL URI will first attempt to match against the uri_aliases list. An application can disable this by using VIR_CONNECT_NO_ALIASES * docs/uri.html.in: Document URI aliases * include/libvirt/libvirt.h.in: Add VIR_CONNECT_NO_ALIASES * libvirt.spec.in, mingw32-libvirt.spec.in: Add /etc/libvirt/libvirt.conf * src/Makefile.am: Install default config file * src/libvirt.c: Add support for URI aliases * src/remote/remote_driver.c: Don't try to handle URIs with no scheme and which clearly are not paths * src/util/conf.c: Don't raise error on virConfFree(NULL) * src/xen/xen_driver.c: Don't raise error on URIs with no scheme 2011-10-19 Daniel P. Berrange Add support for autodestroy of guests to the LXC and UML drivers We recently added support for VIR_DOMAIN_START_AUTODESTROY and an impl to the QEMU driver. It is very desirable to support in other drivers, so this adds it to LXC and UML * src/lxc/lxc_conf.h, src/lxc/lxc_driver.c, src/uml/uml_conf.h, src/uml/uml_driver.c: Wire up autodestroy functions 2011-10-19 Guido Günther Use virBufferEscapeShell in cmdEcho qemu: replace qemuMonitorEscapeShell by virBufferEscapeShell virBufferEscapeShell: Emit quotes for the empty string Make the empty string return '' to match cmdEcho's behavior. 2011-10-19 Eric Blake snapshot: detect when qemu lacks disk-snapshot support Noticed when testing new libvirt against old qemu that lacked the snapshot_blkdev HMP command. Libvirt was mistakenly treating the command as successful, and re-writing the domain XML to use the just-created 0-byte file, rendering the domain broken on restart. * src/qemu/qemu_monitor_text.c (qemuMonitorTextDiskSnapshot): Notice another possible error message. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateSingleDiskActive): Don't keep 0-byte file on failure. 2011-10-18 Guido Günther virBufferEscapeShell: Fix escaping of single quotes. When checking if we need to escape a single quote we were looking at the character after the quote instead of at the quote itself. 2011-10-18 Xu He Jie compile: Add a missing function 'pciDeviceListFind' to libvirt_private.syms compile error: ./src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_hostdev.o): In function `qemuPrepareHostdevPCIDevices': /home/soulxu/data/work-code/libvirt/src/qemu/qemu_hostdev.c:183: undefined reference to `pciDeviceListFind' /home/soulxu/data/work-code/libvirt/src/qemu/qemu_hostdev.c:230: undefined reference to `pciDeviceListFind' ./src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_hostdev.o): In function `qemuGetActivePciHostDeviceList': /home/soulxu/data/work-code/libvirt/src/qemu/qemu_hostdev.c:102: undefined reference to `pciDeviceListFind' ./src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_hostdev.o): In function `qemuDomainReAttachHostdevDevices': /home/soulxu/data/work-code/libvirt/src/qemu/qemu_hostdev.c:370: undefined reference to `pciDeviceListFind' 2011-10-18 Osier Yang qemu: Do not wait if the PCI device is not managed when reattaching Waiting for qemu-kvm cleaning up the PCI bar(s) mapping with long time while the device is not managed is just waste of time. 2011-10-17 Jim Fehlig Add AHCI support to qemu driver Tested with multiple AHCI controllers and multiple disks attached to a controller. E.g.,
2011-10-17 Eric Blake snapshot: implement LIST_LEAVES flag in esx Relatively straight-forward filtering. * src/esx/esx_vi.h (esxVI_GetNumberOfSnapshotTrees) (esxVI_GetSnapshotTreeNames): Add parameter. * src/esx/esx_vi.c (esxVI_GetNumberOfSnapshotTrees) (esxVI_GetSnapshotTreeNames): Allow leaf filtering. * src/esx/esx_driver.c (esxDomainSnapshotNum) (esxDomainSnapshotListNames, esxDomainSnapshotNumChildren) (esxDomainSnapshotListChildrenNames): Pass new flag through. 2011-10-17 Philipp Hahn Fix virt-sanlock-cleanup documentation The referenced page does not exist, but locking.html has a section about sanlock. 2011-10-17 Jiri Denemark Fix VPATH build probes.h is generated in build directory; setting a dependency on probes.h from source directory doesn't work well in VPATH builds. Caused by commit 1afcfbdda0cac112faa61f74ec943e46aa43f2f5 qemu: Relax -no-shutdown check to [0.14.0, 0.15.0] The patch that fixes SIGTERM handling with -no-shutdown was taken into 0.15.1 stable release of qemu. 2011-10-17 Philipp Hahn Fix two comments related to error handling 2011-10-15 Wen Congyang build: fix 'make dist' error When I run 'make dist', I receive the following error messages: make[1]: Entering directory `/home/wency/source/libvirt/src' GEN remote/remote_protocol.h GEN remote/remote_protocol.c GEN remote/qemu_protocol.h GEN remote/qemu_protocol.c GEN remote/qemu_client_bodies.h CC libvirt_driver_remote_la-remote_protocol.lo In file included from ./remote/remote_protocol.h:16, from ./remote/remote_protocol.c:7: /internal.h:249:23: error: probes.h: No such file or directory make[1]: *** [libvirt_driver_remote_la-remote_protocol.lo] Error 1 make[1]: Leaving directory `/home/wency/source/libvirt/src' make: *** [distdir] Error 1 The reason is that we use probes.h before generating it. 2011-10-15 Roopa Prabhu Add missing strdup return value check Check strdup return value and fail if error 2011-10-14 Osier Yang qemu: Honor the orginal PCI dev properties when reattaching BZ# https://bugzilla.redhat.com/show_bug.cgi?id=736214 The problem is caused by the original info of domain's PCI dev is maintained by qemu_driver->activePciHostdevs list, (E.g. dev->reprobe, which stands for whether need to reprobe driver for the dev when do reattachment). The fields (dev->reprobe, dev->unbind_from_stub, and dev->remove_slot) are initialized properly when preparing the PCI device for managed attachment. However, when do reattachment, it construct a complete new "pciDevice" without honoring the original dev info, and thus the dev won't get the original driver or can get other problem. This patch is to fix the problem by get the devs from list driver->activePciHostdevs. Tested with following 3 scenarios: * the PCI was bound to some driver not pci-stub before attaching result: the device will be bound to the original driver * the PCI was bound to pci-stub before attaching result: no driver reprobing, and still bound to pci-stub * The PCI was not bound to any driver result: no driver reprobing, and still not bound to any driver. 2011-10-14 Roopa Prabhu macvtap: avoid invalid free Commit 0472f39 plugged a leak, but introduced another bug: Actually looks like physfndev is conditionally allocated in getPhysfnDev Its better to modify getPhysfnDev to allocate physfndev every time. 2011-10-14 Laine Stump docs: fix network XML documentation A few people have attempted to use the new forwarding modes with older versions of libvirt. The docs where the modes are described have always stated the minimum required libvirt version, but the examples at the end didn't, which I believe is what has caused the confusion. Similarly, the section on portgroups now has a version tag added at the beginning. I also noticed that there was no example of defining a hostname, so I added one, as well as making the domain name example more recognizable (by adding ".com" to the domain). 2011-10-14 Jiri Denemark Clarify semantics of virDomainMigrate2 Explicitly disallow conflicts between domain name from dxml and dname. 2011-10-14 Osier Yang qemu: Do not reattach PCI device used by other domain when shutdown When failing on starting a domain, it tries to reattach all the PCI devices defined in the domain conf, regardless of whether the devices are still used by other domain. This will cause the devices to be deleted from the list qemu_driver->activePciHostdevs, thus the devices will be thought as usable even if it's not true. And following commands nodedev-{reattach,reset} will be successful. How to reproduce: 1) Define two domains with same PCI device defined in the confs. 2) # virsh start domain1 3) # virsh start domain2 4) # virsh nodedev-reattach $pci_device You will see the device will be reattached to host successfully. As pciDeviceReattach just check if the device is still used by other domain via checking if the device is in list driver->activePciHostdevs, however, the device is deleted from the list by step 2). This patch is to prohibit the bug by: 1) Prohibit a domain starting or device attachment right at preparation period (qemuPrepareHostdevPCIDevices) if the device is in list driver->activePciHostdevs, which means it's used by other domain. 2) Introduces a new field for struct _pciDevice, (const char *used_by), it will be set as the domain name at preparation period, (qemuPrepareHostdevPCIDevices). Thus we can prohibit deleting the device from driver->activePciHostdevs if it's still used by other domain when stopping the domain process. * src/pci.h (define two internal functions, pciDeviceSetUsedBy and pciDevceGetUsedBy) * src/pci.c (new field "const char *used_by" for struct _pciDevice, implementations for the two new functions) * src/libvirt_private.syms (Add the two new internal functions) * src/qemu_hostdev.h (Modify the definition of functions qemuPrepareHostdevPCIDevices, and qemuDomainReAttachHostdevDevices) * src/qemu_hostdev.c (Prohibit preparation and don't delete the device from activePciHostdevs list if it's still used by other domain) * src/qemu_hotplug.c (Update function usage, as the definitions are changed) 2011-10-14 Philipp Hahn Xen: Fake versions in xencapstest virInitialize() → xenRegister() → xenhypervisorInit() determines the version of the Hypervisor. This breaks xencapstest when building as root on a dom0 system, since xenHypervisorBuildCapabilities() adds the "hap" and "viridian" features based on the detected version. Add an optional parameter to xenhypervisorInit() to disable automatic detection of the Hypervisor version. The passed in arguments are used instead. 2011-10-14 Philipp Hahn Xen: move versions to struct Calling virInitialize() → xenRegister() → xenhypervisorInit() directly opens a connection to the Xen Hypervisor, which breaks some unit tests. Move all static variables into a struct to make it easier to override them when testing. 2011-10-14 Eric Blake esx: drop dead code to silence Coverity Coverity detected that the only way to get to the cleanup label is if objectSpec had been successfully allocated, so the null check was dead code. * src/esx/esx_vi.c (esxVI_LookupObjectContentByType): Drop redundant null check. 2011-10-14 Jiri Denemark util: Fix typo in virGetHostname description 2011-10-14 Daniel P. Berrange Fix syntax problem in mingw32-libvirt.spec.in When defining macros, you can't put comments on the end of the line because they will get included in the macro definition * mingw32-libvirt.spec.in: Fix comment about hyperv 2011-10-14 Eric Blake macvtap: plug memory leak for 802.1Qbh Detected by Coverity. Leak present since commit ca3b22b. * src/util/macvtap.c (doPortProfileOp8021Qbh): Release device name. 2011-10-14 Eric Blake qemu: plug memory leak on migration Detected by Coverity. Leak introduced in commit 72de0d2. * src/qemu/qemu_migration.c (qemuMigrationCookieGraphicsXMLParse): Clean up on success. 2011-10-14 Eric Blake conf: plug memory leak on error Detected by Coverity. Leak present since commit 874e65a; and while commit d50bb45 tried to fix the issue, it missed a path. * src/conf/domain_conf.c (virDomainDefParseBootXML): Always clean up useserial. 2011-10-13 Guido Günther Use virBufferEscapeShell in virNetSocketNewConnectSSH to escape the netcat command since it's passed to the shell. Adjust expected test case output accordingly. Add virBufferEscapeShell Escape strings so they're safe to pass to the shell. It's based on virsh's cmdEcho. 2011-10-13 Guido Günther Autodetect if the remote nc command supports the -q option Based on a patch by Marc Deslauriers RH: https://bugzilla.redhat.com/show_bug.cgi?id=562176 Ubuntu: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/517478 Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573172 2011-10-13 Eric Blake storage: plug memory leak on error Detected by Coverity. Present since commit 82c1740. * src/storage/storage_backend_logical.c (virStorageBackendLogicalMakeVol): Fix leak. 2011-10-13 Jiri Denemark util: Make getaddrinfo failure nonfatal in virGetHostname Setting a hostname that cannot be resolved is not the best configuration but since virGetHostname only calls getaddrinfo to get host's canonical name and we do not fail if the returned canonical name is NULL or "localhost", there is no reason why we should fail if getaddrinfo itself fails. qemu: Make sure BeginJob is always followed by EndJob Otherwise we can end up with a dangling job that can only be cleared by restarting libvirtd. qemu: Log debug messages when changing job Log debug messages anytime we call *BeginJob* or *EndJob* so that it's easier to spot incorrect usage of domain job APIs. 2011-10-13 Eric Blake qemu: fix text block info parsing Detected by Coverity. p (the pointer to the string) is always true; when in reality, we wanted to know whether the integer value of the just-parsed string is '0' or '1'. Logic bug since commit b1b5b51. * src/qemu/qemu_monitor_text.c (qemuMonitorTextGetBlockInfo): Set results to proper value. 2011-10-13 Eric Blake qemu: avoid text monitor null deref Detected by Coverity. If, for some reason, our text monitor input does not match our assumptions, we end up incrementing p while it is NULL, then dereferencing the pointer 0x1, which will fault. * src/qemu/qemu_monitor_text.c (qemuMonitorTextGetBlockStatsParamsNumber): Rewrite to avoid deref of strchr failure. Fix indentation. 2011-10-13 Eric Blake qemu: check for json allocation failure Detected by Coverity. Introduced in commit b1b5b51. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetBlockInfo): Avoid null dereference. 2011-10-13 Eric Blake build: add compiler attributes to virUUIDParse Coverity complained that most, but not all, clients of virUUIDParse were checking for errors. Silence those coverity warnings by explicitly marking the cases where we trust the input, and fixing one instance that really should have been checking. In particular, this silences a rather large percentage of the warnings I saw on my most recent Coverity analysis run. * src/util/uuid.h (virUUIDParse): Enforce rules. * src/util/uuid.c (virUUIDParse): Drop impossible check; at least Coverity will detect if we break rules and pass NULL. * src/xenapi/xenapi_driver.c (xenapiDomainCreateXML) (xenapiDomainLookupByID, xenapiDomainLookupByName) (xenapiDomainDefineXML): Ignore return when we trust data source. * src/vbox/vbox_tmpl.c (nsIDtoChar, vboxIIDToUUID_v3_x) (vboxCallbackOnMachineStateChange) (vboxCallbackOnMachineRegistered, vboxStoragePoolLookupByName): Likewise. * src/node_device/node_device_hal.c (gather_system_cap): Likewise. * src/xenxs/xen_sxpr.c (xenParseSxpr): Check for errors. 2011-10-13 Marc-André Lureau virFDStream: close also given errfd (fd leak) In virFDStreamOpenFileInternal(), a errfd pipe is opened by virCommandRunAsync() and given to virFDStreamOpenInternal(). It seems virFDStream should close errfd, just like the other fd it is given. This fixes screenshots leaking FDs: http://bugzilla.redhat.com/show_bug.cgi?id=745761 2011-10-13 Eric Blake command: avoid fd leak on failure virCommandTransferFD promises that the fd is no longer owned by the caller. Normally, we want the fd to remain open until the child runs, but in error situations, we must close it earlier. * src/util/command.c (virCommandTransferFD): Close fd now if we can't track it to close later. (virCommandKeepFD): Adjust helper to make this easier. 2011-10-13 Serge E. Hallyn Fix typo in lxc_controller s/Mouting/Mounting. 2011-10-13 Eric Blake spec: mingw cleanups * libvirt.spec.in (%configure): Drop unused %{one} macro. * mingw32-libvirt.spec.in (%{rhel}): Compile ESX but not HyperV on mingw build for RHEL. (%build): Make configure honor spec conditionals. Reorder to match libvirt.spec. * autobuild.sh (mingw): Update list to match. Suggested by Daniel P. Berrange. 2011-10-13 Eric Blake build: update to latest gnulib 358 changes; most probably have no impact on libvirt, but we might as well stay current. * .gnulib: Update to latest. 2011-10-13 Michal Privoznik qemu: Check for domain being active on successful job acquire As this is needed. Although some functions check for domain being active before obtaining job, we need to check it after, because obtaining job unlocks domain object, during which a state of domain can be changed. 2011-10-13 Michal Privoznik events: Propose a separate lock for event queue Currently, push & pop from event queue (both server & client side) rely on lock from higher levels, e.g. on driver lock (qemu), private_data (remote), ...; This alone is not sufficient as not every function that interacts with this queue can/does lock, esp. in client where we have a different approach, "passing the buck". Therefore we need a separate lock just to protect event queue. For more info see: https://bugzilla.redhat.com/show_bug.cgi?id=743817 2011-10-13 Michal Privoznik qemu: Implement VIR_DUMP_RESET This patch extends qemudDomainCoreDump so it supports new VIR_DUMP_RESET flag. If this flag is set, domain is reset on successful dump. However, this is needed to be done after we start CPUs. virDomainCoreDump: Introduce VIR_DUMP_RESET flag This flag is intended to allow user to do so called system reset after dump, instead of sending ACPI reboot event. 2011-10-13 Philipp Hahn example: Support debug output and loop switch Add support for enabling debug output via command line option. Allow to toggle the loop implementation between pure-Python and native-C. example: Redirect --help output to stdout/stderr When --help is requested, print usage() to stdout. When an illegal option is passed, print usage to stderr. example: Fix argument handling sys.argv contains the original command line arguments, while args only contains the arguments not handled by getopt(). Currently this is no problem since --help is the only command line option passable, which terminates the process, so the code is never reached. Any option added in the future will reveal the bug. 2011-10-13 Eric Blake snapshot: implement LIST_LEAVES flag in qemu With the recent refactoring of qemu snapshot relationships, it is now trivial to filter on leaves. * src/conf/domain_conf.c (virDomainSnapshotObjListCount) (virDomainSnapshotObjListCopyNames): Handle new flag. * src/qemu/qemu_driver.c (qemuDomainSnapshotListNames) (qemuDomainSnapshotNum, qemuDomainSnapshotListChildrenNames) (qemuDomainSnapshotNumChildren): Pass new flag through. 2011-10-13 Eric Blake snapshot: add API for filtering by leaves Counterpart to --roots. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_LIST_LEAVES): New flag. * src/libvirt.c (virDomainSnapshotNum, virDomainSnapshotListNames) (virDomainSnapshotNumChildren) (virDomainSnapshotListChildrenNames): Document it. * tools/virsh.c (cmdSnapshotList): Expose it. * tools/virsh.pod (snapshot-list): Document --leaves. 2011-10-12 Philipp Hahn xen: Return tap2 for tap2 disks For some versions of Xen the difference between "tap" and "tap2" is important. When converting back from xen-sxpr to libvirt-xml, that information is lost, which breaks re-defining the domain using that data. Explicitly return "tap2" for disks defined as "device/tap2". 2011-10-12 Philipp Hahn xen: fix PyGrub boot device order When PyGrub is used as the bootloader in Xen, it gets passed the first bootable disk. Xend supports a "bootable"-flag for this, which isn't explicitly supported by libvirt. When converting libvirt-xml to xen-sxpr the "bootable"-flag gets implicitly set by xen.xend.XenConfig.device_add() for the first disk (marked as "Compat hack -- mark first disk bootable"). When converting back xen-sxpr to libvirt-xml, the disks are returned in the internal order used by Xend ignoring the "bootable"-flag, which loses the original order. When the domain is then re-defined, the order of disks is changed, which breaks PyGrub, since a different disk gets passed. When converting xen-sxpr to libvirt-xml, use the "bootable"-flag to determine the first disk. This isn't perfect, since several disks can be marked as bootable using the Xend-API, but that is not supported by libvirt. In all known cases relevant to libvirt exactly one disk is marked as bootable. 2011-10-12 Philipp Hahn tests: Add support for skipping tests AM_TESTS has support for skipping tests, while the C-implementation virtTestRun() does not support that feature. Print "_" or "SKIP" in verbose mode for tests returning EXIT_AM_SKIP=77. 2011-10-12 Harsh Prateek Bora Introduce under to support open-by-handle VirtFS allows the user to choose between path/handle based fs driver. As of now, libvirt hardcoded path based driver only. This patch provides a solution to allow user to choose between path/handle based fs driver. Sample: 2011-10-12 Sage Weil buf: implement generic virBufferEscape Implement a generic helper to escape a given set of characters with a leading '\'. Generalizes virBufferEscapeSexpr(). 2011-10-12 Osier Yang daemon: Always advertise libvirtd service This is a regression introduced by new RPC codes, previously we advertise the service via ssh even if the daemon doesn't listen on TLS port (TCP is not choosed). Now the service is only advertised when it listens on TLS or TCP port. This breaks upper layer apps which intends to discover the service, such as virt-manager. 2011-10-12 Eric Blake snapshot: drop dead parameters The previous optimizations lead to some follow-on cleanups. * src/conf/domain_conf.c (virDomainSnapshotForEachChild) (virDomainSnapshotForEachDescendant): Drop dead parameter. (virDomainSnapshotActOnDescendant) (virDomainSnapshotObjListNumFrom) (virDomainSnapshotObjListGetNamesFrom): Update callers. * src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren) (qemuDomainSnapshotListChildrenNames, qemuDomainSnapshotDelete): Likewise. * src/conf/domain_conf.h: Update prototypes. 2011-10-12 Eric Blake snapshot: take advantage of new relations Among other improvements, virDomainSnapshotForEachDescendant is changed from iterative O(n^2) to recursive O(n). A bit better than the O(n^3) implementation in virsh snapshot-list! * src/conf/domain_conf.c (virDomainSnapshotObjListNum) (virDomainSnapshotObjListNumFrom) (virDomainSnapshotObjeListGetNames, virDomainSnapshotForEachChild) (virDomainSnapshotForEachDescendant): Optimize. (virDomainSnapshotActOnDescendant): Tweak. (virDomainSnapshotActOnChild, virDomainSnapshotMarkDescendant): Delete, now that they are unused. 2011-10-12 Eric Blake snapshot: track qemu snapshot relations Maintain the parent/child relationships of all qemu snapshots. * src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Populate relationships after loading. (qemuDomainSnapshotCreateXML): Set relations on creation; tweak redefinition to reuse existing object. (qemuDomainSnapshotReparentChildren, qemuDomainSnapshotDelete): Clear relations on delete. 2011-10-12 Eric Blake snapshot: framework for more efficient relation traversal No one was using virDomainSnapshotHasChildren, but that was an O(n) function. Exposing and tracking a bit more metadata for each snapshot will allow the same query to be made with an O(1) query of the member field. For single snapshot operations (create, delete), callers can be trusted to maintain the metadata themselves, but for reloading, we can't compute parents as we go since there is no guarantee that parents were parsed before children, so we also provide a function to refresh the relationships, and which can be used to detect if the user has ignored our warnings and been directly modifying files in /var/lib/libvirt/qemu/snapshot. This patch only adds metadata; later patches will actually use it. This layout intentionally hardcodes the size of each snapshot struct, by tracking sibling pointers, rather than having to deal with the headache of yet more memory management by directly sticking a dynamically sized child[] on each parent. * src/conf/domain_conf.h (_virDomainSnapshotObj) (_virDomainSnapshotObjList): Add members. (virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent): New prototypes. (virDomainSnapshotHasChildren): Delete. * src/conf/domain_conf.c (virDomainSnapshotSetRelations) (virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent): New functions. (virDomainSnapshotHasChildren): Drop unused function. * src/libvirt_private.syms (domain_conf): Update exports. 2011-10-12 Eric Blake snapshot: use correct qmp monitor command To date, JSON disk snapshots worked by accident, as they were always using hmp fallback due to a typo in commit e702b5b not picking up on the (intentional) difference in command names between the two monitor protocols. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot): Spell QMP command correctly. Reported by Luiz Capitulino. 2011-10-12 Eric Blake snapshot: virsh shorthand for operating on current snap Rather than having to do: $ virsh snapshot-revert dom $(virsh snapshot-current dom --name) I thought it would be nice to do: $ virsh snapshot-revert dom --current I didn't add 'virsh snapshot-dumpxml --current' since we already have 'virsh snapshot-current' for the same task. snapshot-list accepted a name but did not require it, and that remains the case, with --current serving in place of that name. For all other commands, name used to be required, and can now be replaced by --current; I intentionally made it so that omitting both --current and a name is an error (having the absence of a name imply --current seems just a bit too magic, so --current must be explicit). I also had to keep snapshot-edit backwards-compatible, as the only command that already had a --current argument alongside a name, which still works to both edit a named snapshot and make it current. * tools/virsh.c (vshLookupSnapshot): New helper function. (cmdSnapshotEdit, cmdSnapshotList, cmdSnapshotParent) (cmdSnapshotDelete, cmdDomainSnapshotRevert): Use it, adding an option where needed. * tools/virsh.pod (snapshot-delete, snapshot-edit) (snapshot-list, snapshot-parent, snapshot-revert): Document use of --current. (snapshot-dumpxml): Mention alternative. 2011-10-12 Eric Blake build: fix mingw build without sasl Detected by autogen.sh on a cross-mingw build: Creating library file: .libs/libvirt.dll.a Cannot export virNetSASLContextCheckIdentity: symbol not defined Cannot export virNetSASLContextNewServer: symbol not defined ... * src/libvirt_private.syms (virnetsaslcontext.h): Move symbols... * src/libvirt_sasl.syms: ...to new file. * src/Makefile.am (USED_SYM_FILES) [HAVE_SASL]: Use new file. (EXTRA_DIST): Ship it. 2011-10-11 Eric Blake build: fix 'make rpm' Since commit ddf3bd3, 'make rpm' failed with: RPM build errors: File not found: /home/remote/eblake/rpmbuild/BUILDROOT/libvirt-0.9.6-1.fc14.eblake1318366440.x86_64/usr/share/systemtap/tapset/libvirtd.stp * libvirt.spec.in (with_dtrace): Match installed .stp files. 2011-10-11 Eric Blake build: fix 'make distcheck' I got these distcheck failures with sanlock enabled: ERROR: files left in build directory after distclean: ./tools/virt-sanlock-cleanup ./src/locking/qemu-sanlock.conf * src/Makefile.am (DISTCLEANFILES) [HAVE_SANLOCK]: Clean built file. * tools/Makefile.am (DISTCLEANFILES): Likewise. 2011-10-11 Eric Blake build: ship helper scripts Otherwise, 'make rpm' fails with: GEN libvirt_qemu.def make[2]: *** No rule to make target `dtrace2systemtap.pl', needed by `libvirt_probes.stp'. Stop. * src/Makefile.am (EXTRA_DIST): Add recent script additions. 2011-10-11 Dan Horák disable xenlight for non-Xen platforms when building libvirt in Fedora/s390x I've found that xenlight needs to be explicitly disabled in the spec file. Configure properly sets the library as non-existent, but the %files section still wants to package the 3 /var/*/libvirt/libxl directories. See also https://bugzilla.redhat.com/show_bug.cgi?id=745020 2011-10-11 Eric Blake build: fix 'make check' linkage with dtrace Building on Linux with dtrace enabled was failing 'make check': CCLD nodeinfotest ../src/.libs/libvirt_test.a(libvirt_net_rpc_client_la-virnetclient.o): In function `virNetClientNew': /home/remote/eblake/libvirt/src/rpc/virnetclient.c:162: undefined reference to `libvirt_rpc_client_new_semaphore' On looking further, I see some earlier warnings emitted from libtool: *** Warning: Linking the shared library libvirt.la against the non-libtool *** objects probes.o is not portable! Since src/probes.o is only built on Linux, and even then, only when dtrace is enabled, this failure does not affect other platforms, and despite libtool warning that it is not generally portable, it is not a problem for our use-case in libvirt.la. But it turns out that while libtool is willing to jam raw .o files into an installed shared library (libvirt.la becomes libvirt.so), it is NOT willing to jam the same .o file into the convenience library libvirt_test.la. Perhaps this is a bug in libtool, but even if we get libtool fixed, libvirt must continue to build on platforms with older libtool. So, the fix is the same as we are already using for the libvirt_lxc executable - don't rely on the .o file being in the convenience library, but instead use LDADD to pull it in directly. * tests/Makefile.am (PROBES_O): New macro. (LDADDS): Use it to fix link errors. 2011-10-11 Jiri Denemark build: Fix VPATH build with new probes 2011-10-11 Daniel P. Berrange Fix deps for probes.o to ensure correct build ordering 2011-10-11 Daniel P. Berrange Update examples for probing with systemtap This removes the old example for legacy probes and adds two new scripts demonstrating many of the new probe point facilities. The rpc-monitor.stp script will print out friendly details of all RPC traffic between a libvirt client/server. This is incredibly useful in seeing what RPC calls are being made, and also debugging problems in the RPC protocol code The events.stp script will print out lots of info about the poll event loop, which is useful for debugging event handling problems * examples/systemtap/events.stp, examples/systemtap/rpc-monitor.stp: New examples * examples/systemtap/client.stp: Remove obsolete example 2011-10-11 Daniel P. Berrange Rewrite all the DTrace/SystemTAP probing The libvirtd daemon had a few crude system tap probes. Some of these were broken during the RPC rewrite. The new modular RPC code is structured in a way that allows much more effective tracing. Instead of trying to hook up the original probes, define a new set of probes for the RPC and event code. The master probes file is now src/probes.d. This contains probes for virNetServerClientPtr, virNetClientPtr, virSocketPtr virNetTLSContextPtr and virNetTLSSessionPtr modules. Also add probes for the poll event loop. The src/dtrace2systemtap.pl script can convert the probes.d file into a libvirt_probes.stp file to make use from systemtap much simpler. The src/rpc/gensystemtap.pl script can generate a set of systemtap functions for translating RPC enum values into printable strings. This works for all RPC header enums (program, type, status, procedure) and also the authentication enum The PROBE macro will automatically generate a VIR_DEBUG statement, so any place with a PROBE can remove any existing manual DEBUG statements. * daemon/libvirtd.stp, daemon/probes.d: Remove obsolete probing * daemon/libvirtd.h: Remove probe macros * daemon/Makefile.am: Remove all probe buildings/install * daemon/remote.c: Update authentication probes * src/dtrace2systemtap.pl, src/rpc/gensystemtap.pl: Scripts to generate STP files * src/internal.h: Add probe macros * src/probes.d: Master list of probes * src/rpc/virnetclient.c, src/rpc/virnetserverclient.c, src/rpc/virnetsocket.c, src/rpc/virnettlscontext.c, src/util/event_poll.c: Insert probe points, removing any DEBUG statements that duplicate the info 2011-10-11 Daniel P. Berrange Fix missing lock calls on virNetTLSContextRef The virNetTLSContextRef API forgot to acquire/release the lock while changing ctxt->refs * src/rpc/virnettlscontext.c: Add lock calls 2011-10-11 Daniel P. Berrange Refactor TLS to facilitate dynamic probing Pull the call to gnutls_x509_crt_get_dn up into a higher function so that the 'dname' variable will be available for probe points * src/rpc/virnettlscontext.c: Pull gnutls_x509_crt_get_dn up one level 2011-10-11 Daniel P. Berrange Add virSocketRef API to facilitate dynamic probing Instead of directly manipulating sock->refs, add a virSocketRef API * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virSocketRef 2011-10-11 Daniel P. Berrange If receiving a stream error, mark EOF on the stream If we receive an error on the stream, set the EOF marker so that any further (bogus) incoming data is dropped. * src/rpc/virnetclientstream.c: Set EOF on stream 2011-10-11 Daniel P. Berrange Make libvirt.so include the RPC server code To avoid static linking libvirtd to the RPC server code, which then prevents sane introduction of DTrace probes, put it all in the libvirt.so, and export it * daemon/Makefile.am: Don't link to RPC libraries * src/Makefile.am: Link all RPC libraries to libvirt.so * src/libvirt_private.syms: Export all RPC functions 2011-10-11 Eric Blake snapshot: implement snapshot children listing in esx It was fairly trivial to return snapshot listing based on a point in the hierarchy, rather than starting at all roots. * src/esx/esx_driver.c (esxDomainSnapshotNumChildren) (esxDomainSnapshotListChildrenNames): New functions. 2011-10-11 Eric Blake snapshot: implement snapshot children listing in qemu Not too hard to wire up. The trickiest part is realizing that listing children of a snapshot cannot use SNAPSHOT_LIST_ROOTS, and that we overloaded that bit to also mean SNAPSHOT_LIST_DESCENDANTS; we use that bit to decide which iteration to use, but don't want the existing counting/listing functions to see that bit. * src/conf/domain_conf.h (virDomainSnapshotObjListNumFrom) (virDomainSnapshotObjListGetNamesFrom): New prototypes. * src/conf/domain_conf.c (virDomainSnapshotObjListNumFrom) (virDomainSnapshotObjListGetNamesFrom): New functions. * src/libvirt_private.syms (domain_conf.h): Export them. * src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren) (qemuDomainSnapshotListChildrenNames): New functions. 2011-10-11 Eric Blake snapshot: remote protocol for snapshot children Very mechanical. I'm so glad we've automated the generation of things, compared to what it was in 0.8.x days, where this would be much longer. * src/remote/remote_protocol.x (REMOTE_PROC_DOMAIN_SNAPSHOT_NUM_CHILDREN) (REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES): New rpcs. (remote_domain_snapshot_num_children_args) (remote_domain_snapshot_num_children_ret) (remote_domain_snapshot_list_children_names_args) (remote_domain_snapshot_list_children_names_ret): New structs. * src/remote/remote_driver.c (remote_driver): Use it. * src/remote_protocol-structs: Update. 2011-10-11 Eric Blake snapshot: virsh fallback for snapshot-list --descendants --from Given a list of snapshots and their parents, finding all descendants requires a hairy traversal. This code is O(n^3); it could maybe be made to scale O(n^2) with the use of a hash table, but that costs more memory. Hopefully there aren't too many people with a hierarchy so large as to approach REMOTE_DOMAIN_SNAPSHOT_LIST_NAMES_MAX (1024). * tools/virsh.c (cmdSnapshotList): Add final fallback. 2011-10-11 Eric Blake snapshot: virsh fallback for snapshot-list --from children Iterating over one level of children requires parsing all snapshots and their parents; a bit of code shuffling makes it pretty easy to do this as well. * tools/virsh.c (cmdSnapshotList): Add another fallback. 2011-10-11 Eric Blake snapshot: virsh fallback for snapshot-list --tree --from Emulating --from requires grabbing the entire list of snapshots and their parents, and recursively iterating over the list from the point of interest - but we already do that for --tree. This turns on emulation for that situation. * tools/virsh.c (__vshControl): Rename member. (vshReconnect, cmdConnect, vshGetSnapshotParent): Update clients. (cmdSnapshotList): Add fallback. 2011-10-11 Eric Blake snapshot: virsh snapshot-list and children Sometimes, we only care about one branch of the snapshot hierarchy. Make it easier to list a single branch, by using the new APIs. Technically, I could emulate these new virsh options on old servers by doing a complete dump, then scraping xml to filter out just the snapshots that I care about, but I didn't want to do that in this patch. * tools/virsh.c (cmdSnapshotList): Add --from, --descendants. * tools/virsh.pod (snapshot-list): Document them. 2011-10-11 Eric Blake snapshot: new virDomainSnapshotListChildrenNames API The previous API addition allowed traversal up the hierarchy; this one makes it easier to traverse down the hierarchy. In the python bindings, virDomainSnapshotNumChildren can be generated, but virDomainSnapshotListChildrenNames had to copy from the hand-written example of virDomainSnapshotListNames. * include/libvirt/libvirt.h.in (virDomainSnapshotNumChildren) (virDomainSnapshotListChildrenNames): New prototypes. (VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS): New flag alias. * src/libvirt.c (virDomainSnapshotNumChildren) (virDomainSnapshotListChildrenNames): New functions. * src/libvirt_public.syms: Export them. * src/driver.h (virDrvDomainSnapshotNumChildren) (virDrvDomainSnapshotListChildrenNames): New callbacks. * python/generator.py (skip_impl, nameFixup): Update lists. * python/libvirt-override-api.xml: Likewise. * python/libvirt-override.c (libvirt_virDomainSnapshotListChildrenNames): New wrapper function. 2011-10-11 Eric Blake docs: fix html bug doesn't exist. Introduced in commit 4bb4109f. * docs/formatdomain.html.in: Use correct end tag. 2011-10-10 Guido Günther xen_xs: Guard against set but empty kernel argument On xen 4.1 I observed configurations that look like: (image (hvm (kernel '') (loader '/foo/bar') )) The kernel element is there but unset. This leads to an empty element in the XML and even worse makes us skip the boot order parsing and therefore not emit a element which breaks CD booting. 2011-10-10 Guido Günther xen: add error handling to UUID parsing otherwise a missing UUID in a domain config just shows: error: An error occurred, but the cause is unknown Now we have: error: configuration file syntax error: config value uuid was missing 2011-10-10 Eric Blake maint: typo fixes I noticed a couple typos in recent commits, and fixed the remaining instances of them. * docs/internals/command.html.in: Fix spelling errors. * include/libvirt/libvirt.h.in (virConnectDomainEventCallback): Likewise. * python/libvirt-override.py (virEventAddHandle): Likewise. * src/lxc/lxc_container.c (lxcContainerChild): Likewise. * src/util/hash.c (virHashCreateFull): Likewise. * src/storage/storage_backend_logical.c (virStorageBackendLogicalMakeVol): Likewise. * src/esx/esx_driver.c (esxFormatVMXFileName): Likewise. * src/vbox/vbox_tmpl.c (vboxIIDIsEqual_v3_x): Likewise. 2011-10-10 Eric Blake snapshot: avoid accidental renames with snapshot-edit I was a bit surprised that 'virsh snapshot-edit dom name' silently allowed me to clone things, while still telling me the old name, especially since other commands like 'virsh edit dom' reject rename attempts (*). This fixes things to be more explicit (**). (*) Technically, 'virsh edit dom' relies on virDomainDefineXML behavior, which rejects attempts to mix a new name with existing uuid or new uuid with existing name, but you can create a new domain by changing both uuid and name. On the other hand, while snapshot-edit --clone is a true clone, creating a new domain would also have to decide whether to clone snapshot metadata, managed save, and any other secondary data related to the domain. Domain renames are not trivial either. (**) Renaming or creating a clone is still a risky proposition - for offline snapshots and system checkpoints, if the new name does not match an actual name recorded in the qcow2 internal snapshots, then you cannot revert to the new checkpoint. But it is assumed that anyone using the new virsh flags knows what they are doing, and can deal with the fallout caused by a rename/clone; that is, we can't completely prevent a user from shooting themselves in the foot, so much as we are making the default action less risky. * tools/virsh.c (cmdSnapshotEdit): Add --rename, --clone. * tools/virsh.pod (snapshot-edit): Document them. 2011-10-10 Eric Blake snapshot: sort snapshot-list --tree Otherwise, the results are not repeatable. * tools/virsh.c (cmdSnapshotList): Print tree in predictable order. 2011-10-10 Osier Yang storage: Do not use comma as seperator for lvs output * src/storage/storage_backend_logical.c: If a logical vol is created as striped. (e.g. --stripes 3), the "device" field of lvs output will have multiple fileds which are seperated by comma. Thus the RE we write in the codes will not work well anymore. E.g. (lvs output for a stripped vol, uses "#" as seperator here): test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\ /dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304 The RE we use: const char *regexes[] = { "^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$" }; Also the RE doesn't match the "devices" field of striped vol properly, it contains multiple "device path" and "offset". This patch mainly does: 1) Change the seperator into "#" 2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)" into "(\\S+)". 3) Add two new options for lvs command, (segtype, stripes) 4) Extend the RE to match the value for the two new fields. 5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol, multiple "extents" info are generated if the vol is striped. The number of "extents" is equal to the stripes number of the striped vol. A incidental fix: (virStorageBackendLogicalMakeVol) Free "vol" if it's new created and there is error. Demo on striped vol with the patch applied: % virsh vol-dumpxml /dev/test_vg/vol_striped2 vol_striped2 QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs 8388608 8388608 /dev/test_vg/vol_striped2 0660 0 6 RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474 2011-10-09 Michal Privoznik qemuDomainAttach: Initialize pidfile variable If parsing qemu command line fails (e.g. because of non-existing process number supplied), we jump to cleanup label where we free pidfile. Therefore it needs to be initialized. Otherwise we free random pointer. 2011-10-08 Eric Blake qemu: silence Coverity false positive Coverity complained that 4 out of 5 callers to virJSONValueObjectGetBoolean checked for errors. But we documented that we don't care in this case. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetBlockInfo): Use ignore_value. 2011-10-08 Eric Blake lxc: fix logic bug Detected by Coverity. We want to increment the size_t counter, not the pointer to the counter. Bug present since 5f5c6fde (0.9.5). * src/lxc/lxc_controller.c (lxcSetupLoopDevices): Use correct precedence. 2011-10-07 tangchen virsh: Update the help information for undefine command. virsh undefine command can now undefine an active guest, but the help information is still the old. This patch modifies it and make it coincident to the manpage of virsh. 2011-10-07 Daniel P. Berrange Don't send back unknown program errors for async messages If we send back an unknown program error for async messages, we will confuse the client because they only expect replies for method calls. Just log & drop any invalid async messages * src/rpc/virnetserver.c: Don't send error for async messages 2011-10-07 Daniel P. Berrange Fix deadlock when the RPC program is unknown Commit 597fe3cee68f561a181967b59a87b4e5c5880c4c accidentally introduced a deadlock when reporting an unknown RPC program. The virNetServerDispatchNewMessage method is called with the client locked, and must therefore not attempt to send any RPC messages back to the client. Only once the incoming message is passed off to the virNetServerHandleJob worker is it safe to start sending messages back * src/rpc/virnetserver.c: Delay checking for unknown RPC program until in worker thread 2011-10-07 Eric Blake snapshot: simplify redefinition of disk snapshot Redefining disk-only snapshot xml should work even if the user did not explicitly pass VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY; the flag is only required for conditions where the subelement is not already present in parsing (that is, defining a new snapshot). Also, fix the error code of some user-visible errors (the remaining VIR_ERR_INTERNAL_ERROR should not be user-visible, since parsing of is only done from internal code). * src/conf/domain_conf.c (virDomainSnapshotDefParseString): Allow disks during redefinition of disk snapshot. 2011-10-07 Eric Blake snapshot: let virsh edit disk snapshots It was impossible for 'virsh snapshot-current dom name' to set name as the current snapshot, if name is a disk-only snapshot. Using strstr rather than full-blown xml parsing is safe, since the xml is assumed to be well-formed coming from libvirtd rather than arbitrary text coming from the user. * tools/virsh.c (cmdSnapshotCurrent, cmdSnapshotEdit): Pass disk_only flag when redefining a disk snapshot. 2011-10-07 Eric Blake snapshot: fix virsh error message typo * tools/virsh.c (cmdSnapshotList): Spell exclusive correctly. 2011-10-07 Michal Privoznik remote_driver: Avoid double free in EventControl building Don't xdr_free event data as they are freed by our caller virNetClientProgramDispatch. 2011-10-07 Eric Blake build: fix 'make distcheck' with pdwtags installed I am getting this failure with 'make distcheck': GEN ../../src/remote_protocol-structs /bin/sh: ../../src/remote_protocol-structs-t: Permission denied make[4]: *** [../../src/remote_protocol-structs] Error 1 since it attempts a sub-run of a VPATH 'make check' where $(srcdir) is intentionally read-only. I'm not sure which commit introduced the problem, although I suspect it was around 62dee6f when I refactored protocol struct checking to be more powerful. $(@F) is required by POSIX, and although it is not yet portable to all make implementations, we already require GNU make. * src/Makefile.am (PDWTAGS): Generate temp file into current directory, since $(srcdir) is read-only during distcheck. 2011-10-06 Guido Günther xenParseXM: don't dereference NULL pointer when script is empty 2011-10-06 Laine Stump qemu: add separate rerror_policy for disk errors Previously libvirt's disk device XML only had a single attribute, error_policy, to control both read and write error policy, but qemu has separate options for controlling read and write. In one case (enospc) a policy is allowed for write errors but not read errors. This patch adds a separate attribute that sets only the read error policy. If just error_policy is set, it will apply to both read and write error policy (previous behavior), but if the new rerror_policy attribute is set, it will override error_policy for read errors only. Possible values for rerror_policy are "stop", "report", and "ignore" ("report" is the qemu-controlled default for rerror_policy when error_policy isn't specified). For consistency, the value "report" has been added to the possible values for error_policy as well. 2011-10-06 Laine Stump qemu: leave rerror policy at default when enospace is requested commit 12062ab set rerror=ignore when error_policy="enospace" was selected (since the rerror option in qemu doesn't accept "enospc", as the werror option does). After that patch was already pushed, Paolo Bonzini noticed it and commented that leaving rerror at the default ("report") would be a better choice. This patch corrects the problem - if error_policy = "enospace" is given, rerror is left off the qemu commandline, effectively setting it to "report". For other values, rerror is still set to match werror. Additionally, the parsing of error_policy was changed to no longer erroneously allow "default" as a choice - as with most other attributes, if you want the default setting, just don't specify an error_policy. Finally, two ommissions in the first patch were corrected - a long-dormant qemuxml2argv test for enospace was enabled, and fixed to pass, and the argv2xml parser in qemu_command.c was updated to recognize the different spelling on the qemu commandline. 2011-10-06 Eric Blake qemu: enable multifunction for older qemu Now that RHEL 6.2 Beta is out, it would be nice to test multifunction devices on that platform. This changes things so that the multifunction cap bit can be set in two different ways: by version comparison (needed for qemu 0.13 which lacked a -device query), and by -device query (provided by qemu.git and backported to the RHEL beta build of qemu-kvm which still claims to be a modified 0.12, and therefore needed for RHEL). * src/qemu/qemu_capabilities.c (qemuCapsParseDeviceStr): Allow second method of setting multifunction cap bit. * tests/qemuhelptest.c (mymain): Test it. * tests/qemuhelpdata/qemu-kvm-0.12.1.2-rhel62-beta: New file. * tests/qemuhelpdata/qemu-kvm-0.12.1.2-rhel62-beta-device: Likewise. 2011-10-06 Guido Günther Document STREQ_NULLABLE and STRNEQ_NULLABLE 2011-10-06 Daniel P. Berrange Make LXC work with new network configuration types If using one of the new non-NAT/routed virtual network configurations, the LXC driver would not know how to setup the VETH devices. Adding in calls to setup the "actual" network configuration at VM startup and cleanup when shutting down fixes this. * src/lxc/lxc_driver.c: Setup/cleanup actual net devs 2011-10-05 Eric Blake init: raise default system aio limits https://bugzilla.redhat.com/show_bug.cgi?id=740899 documents that if qemu uses aio=native for its disks, then it consumes 128 aio requests per disk. On a host with multiple guests, this can quickly run out of kernel aio requests with the default aio-max-nr of 65536. Kernel developers have confirmed that there is no up-front cost to raising this limit (a larger limit merely implies that more aio requests can be issued in parallel, which in turn will result in more kernel memory allocation, only if the system really does use that many requests). Since the system default limit prevents 256 disks, which is well within libvirt's current scalability, this patch installs a file to raise the limit and document it in case a system administrator has further cause to tune the limit. The install only works on platforms new enough to source /etc/sysctl.d/* alongside /etc/sysctl.conf (F14 and RHEL 6). * daemon/libvirtd.sysctl: New file. * daemon/Makefile.am (EXTRA_DIST): Ship it. (install-init, uninstall-init): Install it. * libvirt.spec.in (%files): Include it in rpm. 2011-10-05 Eric Blake maint: fix minor issues in virterror public header Consistent use of tabs, fewer long lines, and a typo fix. * include/libvirt/virterror.h: Fix typos, layout. 2011-10-05 Eric Blake snapshot: enforce REVERT_FORCE on qemu Implements the documentation for snapshot revert vs. force. Part of the patch tightens existing behavior (previously, reverting to an old snapshot without was blindly attempted, now it requires force), while part of it relaxes behavior (previously, it was not possible to revert an active domain to an ABI-incompatible active snapshot, now force allows this transition). * src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Check for risky situations, and allow force to get past them. 2011-10-05 Eric Blake snapshot: use qemu-img on disks in use at time of snapshot Once we know which set of disks belong to a snapshot, reverting or deleting that snapshot should visit just those disks, rather than also visiting disks that were hot-plugged in the meantime or skipping disks that were hot-unplugged in the meantime. * src/qemu/qemu_domain.c (qemuDomainSnapshotForEachQcow2): Use snapshot domain details when available. Avoid NULL deref. 2011-10-05 Eric Blake snapshot: add REVERT_FORCE to API Although reverting to a snapshot is a form of data loss, this is normally expected. However, there are two cases where additional surprises (failure to run the reverted state, or a break in connectivity to the domain) can come into play. Requiring extra acknowledgment in these cases will make it less likely that someone can get into an unrecoverable state due to a default revert. Also create a new error code, so users can distinguish when forcing would make a difference, rather than having to blindly request force. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_REVERT_FORCE): New flag. * src/libvirt.c (virDomainRevertToSnapshot): Document it. * include/libvirt/virterror.h (VIR_ERR_SNAPSHOT_REVERT_RISKY): New error value. * src/util/virterror.c (virErrorMsg): Implement it. * tools/virsh.c (cmdDomainSnapshotRevert): Add --force to virsh. * tools/virsh.pod (snapshot-revert): Document it. 2011-10-05 Eric Blake snapshot: implement snapshot roots listing in vbox Commit 9f5e53e introduced the ability to filter snapshots to just roots, but it was never implemented for VBox until now. The VBox implementation prohibits deletion of a snapshot with multiple children. Hence, there can only be at most one root, which is found by searching for the snapshot with a NULL uuid. Prior to 4.0, snapshotGet looked up by UUID, and snapshotFind looked up by name; after that point, snapshotGet disappeared and snapshotFind handles uuid or name. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotNum) (vboxDomainSnapshotListNames): Implement limiting list to root. 2011-10-05 Jiri Denemark qemu: Don't fail virDomainGetInfo if we can't update balloon info Qemu driver tries to update balloon data in virDomainGetInfo and if it can't do so because there is another monitor job running, it just reports what's known in domain def. However, if there was no job running but getting the data from qemu fails, we would fail the whole API. This doesn't make sense. Let's make the failure nonfatal. 2011-10-05 Eric Blake snapshot: simplify esx snapshot name lookup No need to request the parent of a snapshot if we aren't going to use it. * src/esx/esx_vi.c (esxVI_GetSnapshotTreeByName): Make parent optional. * src/esx/esx_driver.c (esxDomainSnapshotCreateXML) (esxDomainSnapshotLookupByName, esxDomainRevertToSnapshot) (esxDomainSnapshotDelete): Simplify accordingly. 2011-10-05 Eric Blake snapshot: implement snapshot roots listing in esx Commit 9f5e53e introduced the ability to filter snapshots to just roots, but it was never implemented for ESX until now. * src/esx/esx_vi.h (esxVI_GetNumberOfSnapshotTrees) (esxVI_GetSnapshotTreeNames): Add parameter. * src/esx/esx_vi.c (esxVI_GetNumberOfSnapshotTrees) (esxVI_GetSnapshotTreeNames): Allow choice of recursion or not. * src/esx/esx_driver.c (esxDomainSnapshotNum) (esxDomainSnapshotListNames): Use it to limit to roots. 2011-10-05 Laine Stump qemu: correct misspelled 'enospc' option, and only use for werror This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=730909 When support for setting the qemu disk error policy to "enospc" was added, it was inadvertently spelled "enospace". This patch corrects that on the qemu commandline (while retaining the "enospace" spelling for libvirt's XML). Also, while examining the qemu source, I found that "enospc" is not allowed for the read error policy, only for write error policy (makes sense). Since libvirt currently only has a single error policy setting, when "enospace" is selected, the read error policy is set to "ignore". 2011-10-04 Eric Blake snapshot: better virsh handling of missing current, parent Previously, virsh 'snapshot-parent' and 'snapshot-current' were completely silent in the case where the code conclusively proved there was no parent or current snapshot, but differed in exit status; this silence caused some confusion on whether the commands worked. Furthermore, commit d1be48f introduced a regression where snapshot-parent would leak output about an unknown function, but only on the first attempt, when talking to an older server that lacks virDomainSnapshotGetParent. This changes things to consistenly report an error message and exit with status 1 when no snapshot exists, and to avoid leaking unknown function warnings when using fallbacks. * tools/virsh.c (vshGetSnapshotParent): Alter signature, to distinguish between real error and missing parent. Don't pollute last_error on success. (cmdSnapshotParent): Adjust caller. Always output message on failure. (cmdSnapshotList): Adjust caller. (cmdSnapshotCurrent): Always output message on failure. 2011-10-04 Guido Günther Document that ff callbacks need to be invoked from a clean stack. Also fix a typo. 2011-10-04 Jiri Denemark qemu: Fix migration with dname Destination libvirtd remembers the original name in the prepare phase and clears it in the finish phase. The original name is used when comparing domain name in migration cookie. 2011-10-04 Daniel P. Berrange Allow passing of command line args to LXC container When booting a virtual machine with a kernel/initrd it is possible to pass command line arguments using the ...args... element in the guest XML. These appear to the kernel / init process in /proc/cmdline. When booting a container we do not have a custom /proc/cmdline, but we can easily set an environment variable for it. Ideally we could pass individual arguments to the init process as a regular set of 'char *argv[]' parameters, but that would involve libvirt parsing the XML text. This can easily be added later, even if we add the env variable now * docs/drvlxc.html.in: Document env variables passed to LXC * src/conf/domain_conf.c: Add to be parsed for guests of type='exe' * src/lxc/lxc_container.c: Set LIBVIRT_LXC_CMDLINE env var 2011-10-04 Daniel P. Berrange Add support for bandwidth filtering on LXC guests Call virBandwidthEnable after creating the LXC veth, so that any bandwidth controls get applied * src/lxc/lxc_driver.c: Enable bandwidth limiting 2011-10-04 Laine Stump network: fill in bandwidth from portgroup for all forward modes This patch is a fix for: https://bugzilla.redhat.com/show_bug.cgi?id=743176 which was discovered by Dan Berrange while making bandwidth configuration work for LXC guests. Background: Although virtportprofile data from a network portgroup is only applicable for direct mode interfaces, the code that copies bandwidth data from the portgroup was also only being executed in the case of direct mode interfaces. The result was that interfaces using traditional virtual networks (forward mode='nat|route|none'), and those using a host bridge for forwarding, would not pick up bandwidth data from a portgroup defined in the network. This patch moves that code outside the conditional, so that bandwidth information is *alway* copied from the appropriate portgroup (unless the definition itself already has bandwidth information, which would take precedence over what's in the portgroup anyway). 2011-10-04 Neil Wilson bridge_driver.c: Fix autoconf setting Code altered so that it is consistent with the associated comment. The 'autoconf' variable is forced to zero. 2011-10-03 Marc-André Lureau Set to NULL members that have been freed to prevent crashes Do not crash if virStreamFinish is called after error. ==11000== Invalid read of size 4 ==11000== at 0x373A8099A0: pthread_mutex_lock (pthread_mutex_lock.c:51) ==11000== by 0x4C7CADE: virMutexLock (threads-pthread.c:85) ==11000== by 0x4D57C31: virNetClientStreamRaiseError (virnetclientstream.c:203) ==11000== by 0x4D385E4: remoteStreamFinish (remote_driver.c:3541) ==11000== by 0x4D182F9: virStreamFinish (libvirt.c:14157) ==11000== by 0x40FDC4: cmdScreenshot (virsh.c:3075) ==11000== by 0x42BA40: vshCommandRun (virsh.c:14922) ==11000== by 0x42ECCA: main (virsh.c:16381) ==11000== Address 0x59b86c0 is 16 bytes inside a block of size 216 free'd ==11000== at 0x4A06928: free (vg_replace_malloc.c:427) ==11000== by 0x4C69E2B: virFree (memory.c:310) ==11000== by 0x4D57B56: virNetClientStreamFree (virnetclientstream.c:184) ==11000== by 0x4D3DB7A: remoteDomainScreenshot (remote_client_bodies.h:1812) ==11000== by 0x4CFD245: virDomainScreenshot (libvirt.c:2903) ==11000== by 0x40FB73: cmdScreenshot (virsh.c:3029) ==11000== by 0x42BA40: vshCommandRun (virsh.c:14922) ==11000== by 0x42ECCA: main (virsh.c:16381) 2011-10-03 Eric Blake snapshot: implement getparent for vbox Built by copying from existing functions. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotGetParent): New function. 2011-10-03 Eric Blake snapshot: implement getparent for esx Pretty easy to paste together compared to existing functions. * src/esx/esx_driver.c (esxDomainSnapshotGetParent): New function. 2011-10-01 Laine Stump qemu: make PCI multifunction support more manual When support for was added for PCI multifunction cards (in commit 9f8baf, first included in libvirt 0.9.3), it was done by always turning on the multifunction bit for all PCI devices. Since that time it has been realized that this is not an ideal solution, and that the multifunction bit must be selectively turned on. For example, see https://bugzilla.redhat.com/show_bug.cgi?id=728174 and the discussion before and after https://www.redhat.com/archives/libvir-list/2011-September/msg01036.html This patch modifies multifunction support so that the multifunction=on option is only added to the qemu commandline for a device if its PCI
definition has the attribute "multifunction='on'", e.g.:
In practice, the multifunction bit should only be turned on if function='0' AND other functions will be used in the same slot - it usually isn't needed for functions 1-7 (although there are apparently some exceptions, e.g. the Intel X53 according to the QEMU source code), and should never be set if only function 0 will be used in the slot. The test cases have been changed accordingly to illustrate. With this patch in place, if a user attempts to assign multiple functions in a slot without setting the multifunction bit for function 0, libvirt will issue an error when the domain is defined, and the define operation will fail. In the future, we may decide to detect this situation and automatically add multifunction=on to avoid the error; even then it will still be useful to have a manual method of turning on multifunction since, as stated above, there are some devices that excpect it to be turned on for all functions in a slot. A side effect of this patch is that attempts to use the same PCI address for two different devices will now log an error (previously this would cause the domain define operation to fail, but there would be no log message generated). Because the function doing this log was almost completely rewritten, I didn't think it worthwhile to make a separate patch for that fix (the entire patch would immediately be obsoleted). 2011-10-01 Laine Stump conf: remove unused VIR_ENUM_DECL While adding a new enum, I noticed a VIR_ENUM_DECL for a type that doesn't exist. There is also of course no matching VIR_ENUM_IMPL for it. 2011-10-01 Marc-André Lureau virsh: do not unlink NULL file error:could not take a screenshot of xp ==6216== Syscall param unlink(pathname) points to unaddressable byte(s) ==6216== at 0x373A0D4937: unlink (syscall-template.S:82) ==6216== by 0x40FD73: cmdScreenshot (virsh.c:3070) ==6216== by 0x42BA0D: vshCommandRun (virsh.c:14920) ==6216== by 0x42EC97: main (virsh.c:16379) ==6216== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==6216== error:Requested operation is not valid: domain is not running 2011-09-30 Serge E. Hallyn lvm storage backend: handle command_names=1 in lvm.conf If the regexes supported (?:pvs)?, then we could handle this by optionally matching but not returning the initial command name. But it doesn't. So add a new char* argument to virStorageBackendRunProgRegex(). If that argument is NULL then we act as usual. Otherwise, if the string at that argument is found at the start of a returned line, we drop that before running the regex. With this patch, virt-manager shows me lvs with command_names 1 or 0. The definitions of PVS_BASE etc may want to be moved into the configure scripts (though given how PVS is found, IIUC that could only happen if pvs was a link to pvs_real), but in any case no sense dealing with that until we're sure this is an ok way to handle it. 2011-09-30 Michal Privoznik qemu: Check for outstanding async job too Currently, qemuDomainGetXMLDesc and qemudDomainGetInfo check for outstanding synchronous job before (eventual) monitor entering. However, there can be already async job set, e.g. migration. 2011-09-29 Peter Krempa qemu: Fix error message mentioning VNC instead of SPICE 2011-09-29 Xu He Jie virsh: Add 'reset' command for virsh remote: Implement 'reset' for remote driver qemu: Implement 'reset' for qemu driver api: Add public api for 'reset' Add new public api for 'reset'. It can reset domain immediately without any guest shutdown. 2011-09-29 Jiri Denemark logging: Add date to log timestamp logging: Do not log timestamp through syslog Syslog puts the timestamp to every message anyway so this removes redundant data. 2011-09-29 Matthias Bolte hyperv: Report an error for acceptable URI schemes with a transport Before, URIs such as hyperv+ssh:// have been declined by the Hyper-V driver resulting in the remote driver trying to connect to an non-existing libvirtd. Now such URIs trigger an error in the yper-V driver suggesting to try again without the transport part in the scheme. 2011-09-29 Matthias Bolte esx: Report an error for acceptable URI schemes with a transport Before, URIs such as esx+ssh:// have been declined by the ESX driver resulting in the remote driver trying to connect to an non-existing libvirtd. Now such URIs trigger an error in the ESX driver suggesting to try again without the transport part in the scheme. 2011-09-29 Douglas Schilling Landgraf formatdomain.html.in: fix tickpolicy there is no option "none": >From libvirt/src/conf/domain_conf.c VIR_ENUM_IMPL(virDomainTimerTickpolicy, VIR_DOMAIN_TIMER_TICKPOLICY_LAST, "delay", "catchup", "merge", "discard"); Replacing with delay. 2011-09-29 Daniel Veillard virsh: describe attach-interface parameter target This patch is based on a improvement suggested by Kazuhiro Kikuchi of Fujitsu, it gives a description of the target parameter for that command * tools/virsh.pod: add description for target parameter of attach-interface 2011-09-29 Daniel Veillard virsh: update man page for cpu_shares parameter The man page suggest that the cpu_shares parameter of schedinfo allows values 0-262144, but the kernel remaps values 0 and 1 to the minimum 2, just document that behaviour: [root@test ~]# cat /cgroup/cpu/libvirt/qemu/cpu.shares 1024 [root@test ~]# echo 0 > /cgroup/cpu/libvirt/qemu/cpu.shares [root@test ~]# cat /cgroup/cpu/libvirt/qemu/cpu.shares 2 [root@test ~]# echo 1 > /cgroup/cpu/libvirt/qemu/cpu.shares [root@test ~]# cat /cgroup/cpu/libvirt/qemu/cpu.shares 2 [root@test ~]# * tools/virsh.pod: update description of the cpu_shares parameter to indicate the values 0 and 1 are automatically changed by the kernel to minimal value 2 2011-09-28 Michal Privoznik qemu: Check for ejected media during startup and migration If the daemon is restarted so we reconnect to monitor, cdrom media can be ejected. In that case we don't want to show it in domain xml, or require it on migration destination. To check for disk status use 'info block' monitor command. 2011-09-28 Alex Jia qemu: add return value check * src/qemu/qemu_migration.c: if 'vmdef' is NULL, the function virDomainSaveConfig still dereferences it, it doesn't make sense, so should add return value check to make sure 'vmdef' is non-NULL before calling virDomainSaveConfig, in addition, in order to debug later, also should record error information into log. 2011-09-28 Eric Blake snapshot: implement getparent in qemu First hypervisor implementation of the new API. Allows 'virsh snapshot-list --tree' to be more efficient. * src/qemu/qemu_driver.c (qemuDomainSnapshotGetParent): New function. 2011-09-28 Eric Blake snapshot: add virsh snapshot-list --tree Reuse the tree listing of nodedev-list, coupled with the new helper function to efficiently grab snapshot parent names, to produce tree output for a snapshot hierarchy. For example: $ virsh snapshot-list dom --tree root1 | +- sibling1 +- sibling2 | | | +- grandchild | +- sibling3 root2 | +- child * tools/virsh.c (cmdSnapshotList): Add --tree. * tools/virsh.pod (snapshot-list): Document it. 2011-09-28 Eric Blake snapshot: refactor virsh snapshot parent computation Make parent computation reusable, using virDomainSnapshotGetParent when possible. * tools/virsh.c (vshGetSnapshotParent): New helper. (cmdSnapshotParent): Use it. 2011-09-28 Eric Blake snapshot: remote protocol for getparent Mostly straight-forward, although this is the first API that returns a new snapshot based on a snapshot rather than a domain. * src/remote/remote_protocol.x (REMOTE_PROC_DOMAIN_SNAPSHOT_GET_PARENT): New rpc. (remote_domain_snapshot_get_parent_args) (remote_domain_snapshot_get_parent_ret): New structs. * src/rpc/gendispatch.pl: Adjust generator. * src/remote/remote_driver.c (remote_driver): Use it. * src/remote_protocol-structs: Update. 2011-09-28 Eric Blake snapshot: new virDomainSnapshotGetParent API Although a client can already obtain a snapshot's parent by dumping and parsing the xml, then doing a snapshot lookup by name, it is more efficient to get the parent in one step, which in turn will make operations that must traverse a snapshot hierarchy easier to perform. * include/libvirt/libvirt.h.in (virDomainSnapshotGetParent): Declare. * src/libvirt.c (virDomainSnapshotGetParent): New function. * src/libvirt_public.syms: Export it. * src/driver.h (virDrvDomainSnapshotGetParent): New callback. 2011-09-28 Eric Blake docs: document node device XML Coupled with the recent virsh nodedev-* doc patch, this should now give a better picture of libvirt node device handling. * docs/formatnode.html.in: Fill in page. 2011-09-28 Eric Blake docs: document virsh nodedev-* commands This section of the man page was completely missing; I stumbled on it when I had no clue that I had to use nodedev-reattach after I was done playing with device passthrough to one of my guests. * tools/virsh.pod (NODEDEV COMMANDS): New section. (attach-device, detach-device): Add cross-references. 2011-09-28 Laine Stump security: properly chown/label bidirectional and unidirectional fifos This patch fixes the regression with using named pipes for qemu serial devices noted in: https://bugzilla.redhat.com/show_bug.cgi?id=740478 The problem was that, while new code in libvirt looks for a single bidirectional fifo of the name given in the config, then relabels that and continues without looking for / relabelling the two unidirectional fifos named ${name}.in and ${name}.out, qemu looks in the opposite order. So if the user had naively created all three fifos, libvirt would relabel the bidirectional fifo to allow qemu access, but qemu would attempt to use the two unidirectional fifos and fail (because it didn't have proper permissions/rights). This patch changes the order that libvirt looks for the fifos to match what qemu does - first it looks for the dual fifos, then it looks for the single bidirectional fifo. If it finds the dual unidirectional fifos first, it labels/chowns them and ignores any possible bidirectional fifo. (Note commit d37c6a3a (which first appeared in libvirt-0.9.2) added the code that checked for a bidirectional fifo. Prior to that commit, bidirectional fifos for serial devices didn't work because libvirt always required the ${name}.(in|out) fifos to exist, and qemu would always prefer those. 2011-09-28 Jiri Denemark qemu: Preserve fakeReboot flag in domain status Thus, when libvirtd is restarted, it will know if a domain is supposed to be killed or reset when it shuts down. qemu: Finish domain shutdown on reconnect If a domain started with -no-shutdown shuts down while libvirtd is not running, it will be seen as paused when libvirtd reconnects to it. Use the paused reason to detect if a domain was stopped because of shutdown and finish the process just as if a SHUTDOWN event is delivered from qemu. qemu: Check domain status details when reconnecting monitor Current qemu is able to give us detailed domain status (not just if it is running or not) which we can translate into a status reason. qemu: Always remove domain object if MigratePrepare fails If migration failed in Prepare phase after virDomainAssignDef and before a job was started, the domain object was not properly removed. 2011-09-28 Jamie Strandboge fix AppArmor driver for pipe character devices The AppArmor security driver adds only the path specified in the domain XML for character devices of type 'pipe'. It should be using .in and .out. We do this by creating a new vah_add_file_chardev() and use it for char devices instead of vah_add_file(). Also adjust valid_path() to accept S_FIFO (since qemu chardevs of type 'pipe' use fifos). This is https://launchpad.net/bugs/832507 2011-09-28 Peter Krempa virsh: Allow using complete elements with cpu-baseline This patch cleans the cpu baseline function using new libvirt helper functions and fixes XPath expression that selects elements from the source file, that can contain concatenated XMLs, domain XMLs and bare elements. The fixed XPath expression ensures not to select NUMA virsh: Allow using domain and capabilities XMLs with cpu-compare This patch adds extraction of the element from capabilities and domain definition XML documents to improve user experience. https://bugzilla.redhat.com/show_bug.cgi?id=731151 2011-09-28 Eric Blake snapshot: fix man page typos pod2man from perl-5.8.8 (RHEL 5) errors out on ill-formed POD: *** ERROR: unterminated I<...> at line 1114 in file virsh.pod *** ERROR: unterminated I<...> at line 1851 in file virsh.pod Newer pod2man appears to be more tolerant (which is a shame, because it meant that this error is harder to detect). * tools/virsh.pod (undefine, snapshot-current): Add missing >. 2011-09-27 Laine Stump qemu: add ability to set PCI device "rombar" on or off This patch was made in response to: https://bugzilla.redhat.com/show_bug.cgi?id=738095 In short, qemu's default for the rombar setting (which makes the firmware ROM of a PCI device visible/not on the guest) was previously 0 (not visible), but they recently changed the default to 1 (visible). Unfortunately, there are some PCI devices that fail in the guest when rombar is 1, so the setting must be exposed in libvirt to prevent a regression in behavior (it will still require explicitly setting in the guest XML). rombar is forced on/off by adding: inside a element that defines a PCI device. It is currently ignored for all other types of devices. At the moment there is no clean method to determine whether or not the rombar option is supported by QEMU - this patch uses the advice of a QEMU developer to assume support for qemu-0.12+. There is currently a patch in the works to put this information in the output of "qemu-kvm -device pci-assign,?", but of course if we switch to keying off that, we would lose support for setting rombar on all the versions of qemu between 0.12 and whatever version gets that patch. 2011-09-27 Eric Blake spec: F15 still uses cgconfig, RHEL lacks hyperv Commit ecd8725c dropped attempts to probe the cgconfig service on new enough Fedora where systemd took over that aspect of the system, but mistakenly used F14 instead of F15 as the cutoff point. https://bugzilla.redhat.com/show_bug.cgi?id=741358 Also, RHEL does not include HyperV support yet. * libvirt.spec.in (with_cgconfig): Check cgconfig service in F15. (%{?rhel}): Provide default for with_hyperv. 2011-09-27 Jiri Denemark qemu: Relax -no-shutdown check to [0.14.0, 0.15.50) SIGTERM handling for -no-shutdown is already fixed in qemu git and libvirt can safely use it. The downside is that 0.15.50 version of qemu can be any qemu compiled from git, even that without the fix for SIGTERM. However, I think this patch is worth it since excluding 0.15.50 from the check makes testing current qemu with libvirt much easier and someone running qemu from git should be able to rebuild fixed qemu from git if they hit the problem with a hang on shutdown. virsh: Better document --copy-storage migrate options Both --copy-storage-{all,inc} options require disk images to be present on destination host. virsh: Enhance documentation of commands starting jobs Some virsh commands start a (long-running) job that can be monitored using domjobinfo and aborted with domjobabort. Let's be explicit about this in virsh man page. 2011-09-27 Michal Privoznik virLockManagerNopInit: Rename flags to unused_flags As these might be not used and make syntax-check complains about checking them via virCheckFlags. 2011-09-27 Peter Krempa daemon: Don't remove pidfiles in init scripts Init scripts removed pid file of the daemon. Removing pid files may be harmful as new api for crash-safe pidfiles is used (introduced by c8a3a26). 2011-09-27 Michal Privoznik debug: Annotate some variables as unused as they are not used with debugging turned off. 2011-09-27 Osier Yang storage: Do not break the whole vol lookup process in the middle * src/storage/storage_driver.c: As virStorageVolLookupByPath lookups all the pool objs of the drivers, breaking when failing on getting the stable path of the pool will just breaks the whole lookup process, it can cause the API fails even if the vol exists indeed. It won't get any benefit. This patch is to fix it. 2011-09-23 Peter Krempa maint: update authors 2011-09-23 Oskari Saarenmaa Add unsafe cache mode support for disk driver QEMU 0.13 introduced cache=unsafe for -drive, this patch exposes it in the libvirt layer. * Introduced a new QEMU capability flag ($prefix_CACHE_UNSAFE), as even if $prefix_CACHE_V2 is set, we can't know if unsafe is supported. * Improved the reliability of qemu cache type detection. 2011-09-23 Daniel P. Berrange Fix synchronous reading of stream data commit 984840a2c292402926ad100aeea33f8859ff31a9 removed the notification of waiting calls when VIR_NET_CONTINUE messages arrive. This was to fix the case of a virStreamAbort() call being prematurely notified of completion. The problem is that sometimes there are dummy calls from a virStreamRecv() call waiting that *do* need to be notified. These dummy calls should have a status VIR_NET_CONTINUE. So re-add the notification upon VIR_NET_CONTINUE, but only if the waiter also has a status of VIR_NET_CONTINUE. * src/rpc/virnetclient.c: Notify waiting call if stream data arrives * src/rpc/virnetclientstream.c: Mark dummy stream read packet with status VIR_NET_CONTINUE 2011-09-23 Michal Privoznik selinux: Correctly report warning if virt_use_nfs not set Previous patch c9b37fee tried to deal with virt_use_nfs. But setfilecon() returns EOPNOTSUPP on NFS so we need to move the warning to else branch. 2011-09-22 Peter Krempa virsh: Improve virsh manual for virsh memtune command Commit 0a22f54 added --min-guarantee option for the memtune command. This option is supported only by the ESX hypervisor. This patch adds a statement about this fact, to prevent user confusion. This patch also adds explanation how to clear/set to unlimited the memory tunables. (documments the -1 value). 2011-09-22 Osier Yang virsh: Do not ignore the specified flags for cmdSaveImageDefine Introduced by commit 42c52d53c, which added the support for new flags, but forgot to update the API use to pass the flags. 2011-09-22 Michal Privoznik qemu: Transfer inactive XML among cookie If a domain has inactive XML we want to transfer it to destination when migrating with VIR_MIGRATE_PERSIST_DEST. In order to harm the migration protocol as least as possible, a optional cookie was chosen. 2011-09-22 Daniel Veillard Release of libvirt-0.9.6 2011-09-22 Eric Blake snapshot: also delete empty directory The previous patch removed all snapshots, but not the directory where the snapshots lived, which is still a form of stale data. * src/qemu/qemu_domain.c (qemuDomainRemoveInactive): Wipe any snapshot directory. 2011-09-22 Eric Blake snapshot: remove snapshot metadata on transient exit Commit 282fe1f0 documented that transient domains will auto-delete any snapshot metadata when the last reference to the domain is removed, and that management apps are in charge of grabbing any snapshot metadata prior to that point. However, this was not actually implemented for qemu until now. * src/qemu/qemu_driver.c (qemudDomainCreate) (qemuDomainDestroyFlags, qemuDomainSaveInternal) (qemudDomainCoreDump, qemuDomainRestoreFlags, qemudDomainDefine) (qemuDomainUndefineFlags, qemuDomainMigrateConfirm3) (qemuDomainRevertToSnapshot): Clean up snapshot metadata. * src/qemu/qemu_migration.c (qemuMigrationPrepareAny) (qemuMigrationPerformJob, qemuMigrationPerformPhase) (qemuMigrationFinish): Likewise. * src/qemu/qemu_process.c (qemuProcessHandleMonitorEOF) (qemuProcessReconnect, qemuProcessReconnectHelper) (qemuProcessAutoDestroyDom): Likewise. 2011-09-22 Eric Blake snapshot: prepare to remove transient snapshot metadata This patch is mostly code motion - moving some functions out of qemu_driver and into qemu_domain so they can be reused by multiple qemu_* files (since qemu_driver.h must not grow). It also adds a new helper function, qemuDomainRemoveInactive, which will be used in the next patch. * src/qemu/qemu_domain.h (qemuFindQemuImgBinary) (qemuDomainSnapshotWriteMetadata, qemuDomainSnapshotForEachQcow2) (qemuDomainSnapshotDiscard, qemuDomainSnapshotDiscardAll) (qemuDomainRemoveInactive): New prototypes. (struct qemu_snap_remove): New struct. * src/qemu/qemu_domain.c (qemuDomainRemoveInactive) (qemuDomainSnapshotDiscardAllMetadata): New functions. (qemuFindQemuImgBinary, qemuDomainSnapshotWriteMetadata) (qemuDomainSnapshotForEachQcow2, qemuDomainSnapshotDiscard) (qemuDomainSnapshotDiscardAll): Move here... * src/qemu/qemu_driver.c (qemuFindQemuImgBinary) (qemuDomainSnapshotWriteMetadata, qemuDomainSnapshotForEachQcow2) (qemuDomainSnapshotDiscard, qemuDomainSnapshotDiscardAll): ...from here. (qemuDomainUndefineFlags): Update caller. * src/conf/domain_conf.c (virDomainRemoveInactive): Doc fixes. 2011-09-22 Eric Blake snapshot: fix logic bug in qemu undefine Commit 19f8c98 introduced VIR_DOMAIN_UNDEFINE_SNAPSHOTS_METADATA, with the intent that omitting the flag makes undefine fail, and including the flag deletes metadata. But it used the wrong logic. Also, hoist the transient domain sooner, so that we don't accidentally remove metadata of a transient domain. * src/qemu/qemu_driver.c (qemuDomainUndefineFlags): Check correct flag value. 2011-09-22 Eric Blake sanlock: fix memory leak Detected by Coverity. The only way to get to error_unlink is if path was successfully assigned, so the if was useless. Meanwhile, there was a return statement that did not free path. * src/locking/lock_driver_sanlock.c (virLockManagerSanlockSetupLockspace): Fix mem-leak, and drop useless if. 2011-09-22 Eric Blake virsh: fix regression in argv parsing Prior to commit 85d2810, we had an issue where: snapshot-create-as dom name --diskspec spec --diskspec spec failed to parse the second spec, because the first spec had marked that option as no longer requiring an argument. In commit 85d2810, I fixed it by making argv options no longer mark the option as seen. But this in turn breaks mandatory argv options, which now complain that the argv option is missing. This patch reverts that part of 85d2810, and instead replaces it with fixes to no longer clear opts_need_arg of an argv argument. * tools/virsh.c (vshCmddefGetOption, vshCmddefGetData) (vshCommandParse): Fix option parsing for required argv option. (vshCmddefOptParse): Check that argv option is last. * tests/virsh-optparse: Enhance test. 2011-09-22 Osier Yang virsh: More friendly err if no pool is specified for looking up a vol There are 3 ways to lookup a volume, only virStorageVolLookupByName needs pool object. So if no --pool is specified, it will tries to get the volume via virStorageVolLookupByPath/virStorageVolLookupByKey. But if all 3 ways fails, and no --pool is specified, a friendly error might help the user get right way quickly. 2011-09-22 Osier Yang storage: Wait udev events are handled before removing lvm vol Related #BZ: https://bugzilla.redhat.com/show_bug.cgi?id=702260. There are two problems described in the BZ: 1) "Can't remove open logical volume". 2) "Unable to deactivate logical volume "foo"" This patch just intends to fix 2), as 1) is expected if the vol is still used by something, and you never known if "lvchange -an" will fail or not either (sometime, it will succeed, sometimes not). We'd better not look for trouble, :-) For 2), that's caused by race between lvremove and udev event handling, the only workable way now is to wait the events handling are finished, though it might introduce latencies, as "udevadmin settle" exits after *all* events are handled, it's the only way we can fix the racing in libvirt layer. See https://bugzilla.redhat.com/show_bug.cgi?id=570359 for more details. 2011-09-21 Alex Jia qemu: avoid dereferencing a NULL pointer * src/qemu/qemu_process.c: Taking if (qemuDomainObjEndJob(driver, obj) == 0) true branch then 'obj' is NULL, virDomainObjIsActive(obj) and virDomainObjUnref(obj) will dereference NULL pointer. 2011-09-21 Oskari Saarenmaa tests: improve test failure diagnosis * qemuhelptest prints test case name on failure. 2011-09-21 Jiri Denemark qemu: Avoid loop of fake reboots Once virDomainReboot is called for a domain, guest OS initiated shutdown would always result in reboot instead of shutdown. Only virDomainShutdown would actually shutd such domain down. That's because we forgot to reset fakeReboot flag once we asked the domain to reboot. 2011-09-21 Jiri Denemark qemu: Fix shutdown regression with buggy qemu The commit that prevents disk corruption on domain shutdown (96fc4784177ecb70357518fa863442455e45ad0e) causes regression with QEMU 0.14.* and 0.15.* because of a regression bug in QEMU that was fixed only recently in QEMU git. The affected versions of QEMU do not quit on SIGTERM if started with -no-shutdown, which we use to implement fake reboot. Since -no-shutdown tells QEMU not to quit automatically on guest shutdown, domains started using the affected QEMU cannot be shutdown properly and stay in a paused state. This patch disables fake reboot feature on such QEMU by not using -no-shutdown, which makes shutdown work as expected. However, virDomainReboot will not work in this case and it will report "Requested operation is not valid: Reboot is not supported with this QEMU binary". 2011-09-21 Osier Yang API: prefer to use NULLSTR macro 2011-09-21 Eric Blake remote: fix crash on OOM Bug introduced in commit 675464b. On an OOM, this would try to dereference a char* and free the contents as a pointer, which is doomed to failure. Adding a syntax check will prevent mistakes like this in the future. * cfg.mk (sc_prohibit_internal_functions): New syntax check. (exclude_file_name_regexp--sc_prohibit_internal_functions): Add exemptions. * daemon/remote.c (remoteRelayDomainEventIOError) (remoteRelayDomainEventIOErrorReason) (remoteRelayDomainEventGraphics, remoteRelayDomainEventBlockJob): Use correct free function. 2011-09-21 Eric Blake xen: use typical allocations The next patch will add a syntax check that flags this usage in xen as awkward - while it was valid memory management, it was very hard to maintain. Swapping to a more traditional allocation may be a bit slower, but easier to understand. * src/xen/xend_internal.c (xenDaemonListDomainsOld): Use two-level allocation, rather than abusing allocation function. (xenDaemonLookupByUUID): Update caller. 2011-09-21 Osier Yang virsh: Remove useless codes of cmdVolPath Variable "name" is never used in the codes, it's useless. 2011-09-21 Eric Blake build: silence warning on 32-bit build gcc warns when building libvirt 0.9.5 on a 32-bit machine: qemu/qemu_migration.c: In function 'qemuMigrationToFile': qemu/qemu_migration.c:2727:38: error: large integer implicitly truncated to unsigned type [-Woverflow] * src/qemu/qemu_domain.h (QEMU_DOMAIN_FILE_MIG_BANDWIDTH_MAX): Cap to long when building for 32-bit platform. 2011-09-20 Daniel Veillard Release of libvirt-0.9.5 * configure.ac docs/news.html.in libvirt.spec.in: update for the release * po/*.po*: fetch updated translations from transifex and rebuilt 2011-09-20 Daniel Veillard Fix crash on events due to allocation errors remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError, remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics were using const string directly in rpc structure, before calling remoteDispatchDomainEventSend(). But that routine now frees up all the pointed allocated memory from the rpc structure and we end up with a double free. This now strdup() all the strings passed and provide mem_error goto labels to be used when an allocation error occurs. Note that the cleanup isn't completely finished because all relaying function also call make_nonnull_domain() which also allocate a string and never handle the error case. This patches doesn't try to address this as this is only error correctness a priori and touches far more functions in this module: * daemon/remote.c: fix string allocations and memory error handling for remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError, remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics 2011-09-20 Daniel P. Berrange Update to require sanlock 1.8 for license compliance Inexplicably the sanlock code all got placed under the GPLv2-only, so libvirt's use of sanlock introduces a license incompatibility. The sanlock developers have now rearranged the code such that there is a 'sanlock_client.so' which is LGPLv2+ while their daemon remains GPLv2-only. To use the new client library we need to call the new sanlock_init and sanlock_align APIs instead of sanlock_direct_init and sanlock_direct_align. These APIs calls are now routed via the sanlock daemon, instead of doing direct I/O calls to disk. For all this we require sanlock >= 1.8 * configure.ac: Check for sanlock_client.so instead of sanlock.so and fix various comments * libvirt.spec.in: Mandate sanlock >= 1.8 * src/Makefile.am: Link to -lsanlock_client * src/locking/lock_driver_sanlock.c: Use sanlock_init and sanlock_align 2011-09-20 Osier Yang conf: Assign newDef of active domain as persistent conf if it is NULL Libvirt loads the domain conf from status XML if it's running when starting up. The problem is there is no record of the original conf. (dom->newDef is NULL here). So libvirt won't be able to restore the domain conf to original one when destroying/shutdown. E.g. 1) attach a device without "--persistent" 2) restart libvirtd 3) destroy domain 4) start domain One will see the the disk still exists. This patch is to fix the peoblem by assigning persistent domain conf to dom->newDef if it's NULL and the domain is running. 2011-09-20 Osier Yang storage: Ensure the device path exists before refreshing disk pool Doing libvirt_parthelper on an not existed device path will get an unfriendly error message. This patch is to prohibit it. daemon: Error and exit if specified value for timeout is not valid Silently setting "timeout" as -1 if the specified value is invalid is a bit confused. 2011-09-20 Peter Krempa Remove devname identifier from autogenerated RPC code Patch 79cf07a missed one instance of "devname" in source for RPC code generator. 2011-09-20 Eric Blake virsh: tweak previous domblkstat patch Translators are likely to botch trailing spacing; by doing the formatting outside of the translation, we can generally get better alignment. Also, for consistency, use 'bytes read' to match 'bytes written'. * tools/virsh.c (domblkstat_output): Drop trailing space. Tweak rd_bytes output. (cmdDomblkstat, DOMBLKSTAT_LEGACY_PRINT): Update formatting. 2011-09-20 Peter Krempa virsh: doc: Fix supported driver types for attach-disk command Virsh man page lists driver types to be used with attach-device command, but does not specify that those are usable only with the XEN Hypervisor. This patch adds statement, that those options specified are applicable only on the Xen hypervisor and adds option usable with qemu emulator. This patch also changes type of error returned by QEMU driver if the user specifies incompatible driver type from VIR_ERR_INTERNAL_ERROR to VIR_ERR_CONFIG_UNSUPPORTED. 2011-09-20 Eric Blake maint: Prefer names over email in 'git shortlog' Alex recently committed some patches with just an email instead of a preferred name; this fixes things so 'git shortlog' gives nicer output. * .mailmap: Update. 2011-09-19 Peter Krempa virsh: Add more human-friendly output of domblkstat command Users of virsh complain that output of the domblkstat command is not intuitive enough. This patch adds explanation of fields returned by this command to the help section for domblkstat and the man page of virsh. Also a switch --human is added for domblkstat that prints the fields with more descriptive texts. This patch also changes sequence of the output fields and their names back to the order and spelling established by previous versions of virsh to maintain compatibility with scripts. Example of ordered and "translated" output: PRE-patch: virsh # domblkstat 1 vda vda wr_bytes 5170176 vda wr_operations 511 vda rd_bytes 82815488 vda rd_operations 3726 POST-patch: virsh # domblkstat 1 vda vda rd_req 3726 vda rd_bytes 82815488 vda wr_req 478 vda wr_bytes 4965376 Example of human readable output: virsh # domblkstat 1 vda --human Device: vda number of read operations: 3726 number of read bytes: 82815488 number of write operations: 478 number of bytes written: 4965376 https://bugzilla.redhat.com/show_bug.cgi?id=731656 2011-09-19 ajia@redhat.com vmx: avoid memory leak * src/vmx/vmx.c: fix memory leak, 'def' has a initial value 'NULL', so 'goto cleanup' is perfected instead of adding a virConfFree before 'return NULL'. 2011-09-19 ajia@redhat.com util: avoid memory leak Leak in pciGetVirtualFunctionIndex present since commit 17d64ca. * src/util/pci.c: fix memory leak. 2011-09-19 ajia@redhat.com remote: avoid memory leak Leak present since introduction of remoteDomainBuildEventGraphics in commit 987e31e. * src/remote/remote_driver.c: fix memory leak. 2011-09-19 ajia@redhat.com qemu: avoid memory leak Leak introduced in commit 036ad50, affects only error case. * src/qemu/qemu_command.c: fix memory leak. 2011-09-19 ajia@redhat.com daemon: avoid memory leak Introduced in commit efa7fc9f. * daemon/remote.c: fix memory leak in remoteDispatchDomainBlockStatsFlags 2011-09-17 Eric Blake spec: silence warnings when installing in F16 https://bugzilla.redhat.com/show_bug.cgi?id=738725 documents that 'yum install libvirt' in Fedora 16 is rather noisy. This fixes the problems. * libvirt.spec.in (%post client): Silence chkconfig warning about SysV services. (%post) [with_cgconfig]: Drop for Fedora 15 and newer, where systemd does this automatically. 2011-09-17 Eric Blake snapshot: allow disk snapshots of qcow2 disks For all types of disks other than qcow2, we were requesting that SELinux labeling visit the new file as if it were qcow2, which means labeling would try to find the backing files of an empty file. And for a pre-existing qcow2 disk, we were passing NULL, which meant that labelling tried to probe the file type (and if probing is disabled, per the default qemu.conf, this made snapshots fail). What we really want is to make SELinux labeling visit the new file as raw; it will later be converted to qcow2 if qemu successfully made the snapshot. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateSingleDiskActive): Force SELinux labeling to avoid probe of new file. 2011-09-17 Eric Blake snapshot: affect persistent xml after disk snapshot For external snapshots to be useful on persistent domains, we must alter the persistent definition alongside the running definition. Thanks to the possibility of disk hotplug as well as of edits that only affect the persistent xml, we can't assume that vm->def and vm->newDef have the same disk at the same index, so we can only update the persistent copy if the device destination matches up. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateDiskActive) (qemuDomainSnapshotCreateSingleDiskActive): Also affect newDef, if present. 2011-09-17 Eric Blake build: work around lack of MacOS fdatasync Mingw lacks fsync, but gnulib provides that. Meanwhile, gnulib does not (yet) provide fdatasync, so this is a quick hack to fake that function on MacOS X; we can revert this configure change once gnulib gives us a real module. We have been implicitly relying on gnulib's largefile module being pulled in by other modules, but it's better to make that explicit. * bootstrap.conf (gnulib_modules): Add fsync. Make largefile use explicit. * configure.ac (AC_CHECK_FUNCS_ONCE): Check for fdatasync, and fake it with fsync when not present. 2011-09-16 Daniel P. Berrange Prevent crash from dlclose() of libvirt.so When libvirt calls virInitialize it creates a thread local for the virErrorPtr storage, and registers a callback to cleanup memory when a thread exits. When libvirt is dlclose()d or otherwise made non-resident, the callback function is removed from memory, but the thread local may still exist and if a thread later exists, it will invoke the callback and SEGV. There may also be other thread locals with callbacks pointing to libvirt code, so it is in general never safe to unload libvirt.so from memory once initialized. To allow dlclose() to succeed, but keep libvirt.so resident in memory, link with '-z nodelete'. This issue was first found with the libvirt CIM provider, but can potentially hit many of the dynamic language bindings which all ultimately involve dlopen() in some way, either on libvirt.so itself, or on the glue code for the binding which in turns links to libvirt * configure.ac, src/Makefile.am: Ensure libvirt.so is linked with -z nodelete * cfg.mk, .gitignore, tests/Makefile.am, tests/shunloadhelper.c, tests/shunloadtest.c: A test case to unload libvirt while a thread is still running. 2011-09-16 Alex Jia qemu: add return value check * src/qemu/qemu_command.c: missing return value check. 2011-09-16 Jiri Denemark qemu: Introduce shutdown reason for paused state Qemu sends STOP event as part of the shutdown process. Detect such STOP event and consider shutdown to be reason of emitting such event. That's the best we can do until qemu provides us the reason directly in STOP event. This allows us to report shutdown reason for paused state so that apps can detect domains that failed to finish the shutdown process (e.g., because qemu is buggy and doesn't exit on SIGTERM or it is blocked in flushing disk buffers). 2011-09-16 Jiri Denemark qemu: Prevent disk corruption on domain shutdown Ever since we introduced fake reboot, we call qemuProcessKill as a reaction to SHUTDOWN event. Unfortunately, qemu doesn't guarantee it flushed all internal buffers before sending SHUTDOWN, in which case killing the process forcibly may result in (virtual) disk corruption. By sending just SIGTERM without SIGKILL we give qemu time to to flush all buffers and exit. Once qemu exits, we will see an EOF on monitor connection and tear down the domain. In case qemu ignores SIGTERM or just hangs there, the process stays running but that's not any different from a possible hang anytime during the shutdown process so I think it's just fine. Also qemu (since 0.14 until it's fixed) has a bug in SIGTERM processing which causes it not to exit but instead send new SHUTDOWN event and keep waiting. I think the best we can do is to ignore duplicate SHUTDOWN events to avoid a SHUTDOWN-SIGTERM loop and leave the domain in paused state. 2011-09-16 Jiri Denemark qemu: Properly detect crash of a rebooted domain When a domain is rebooted using libvirt API, we use fake reboot consisting of shutting down and resetting the domain. Thus we see a SHUTDOWN event and set gotShutdown flag. But we never reset it back and if the domain crashes after it was rebooted this way, we consider it was a normal shutdown and not a crash. qemu: Fix shutoff reason when domain crashes Commit 4454a9efc728b91e791b1f14c26ea23a19d57f48 changed shutoff reason from VIR_DOMAIN_SHUTOFF_CRASHED to VIR_DOMAIN_SHUTOFF_FAILED in case we see an unexpected EOF on monitor connection. But FAILED reason is dedicated for domains that fail to start. CRASHED reason is the right one to use in this situation. python: Fix bindings generated in VPATH build 2011-09-16 Eric Blake Revert "virsh: Use old API if remote libvirtd does not support new" This reverts commit 799912fa05b8c3aa37bd04c57b196755f3f70552; now that the rpc regression is fixed, virsh no longer needs the special case here. 2011-09-16 Eric Blake rpc: convert unknown procedures to VIR_ERR_NO_SUPPORT Libvirt special-cases a specific VIR_ERR_RPC from the remote driver back into VIR_ERR_NO_SUPPORT on the client, so that clients can handle missing rpc functions the same whether the hypervisor driver is local or remote. However, commit c1b22644 introduced a regression: VIR_FROM_THIS changed from VIR_FROM_REMOTE to VIR_FROM_RPC, so the special casing no longer works if the server uses the newer error domain. * src/rpc/virnetclientprogram.c (virNetClientProgramDispatchError): Also cater to 0.9.3 and newer. 2011-09-16 Stefan Berger Fix buzzilla 738778 This patch fixes the bug shown in bugzilla 738778. It's not an nwfilter problem but a connection sharing / closure issue. https://bugzilla.redhat.com/show_bug.cgi?id=738778 Depending on the speed / #CPUs of the machine you are using you may not see this bug all the time. 2011-09-16 Alex Jia conf: avoid memory leak on virDomainDefParseXML * conf/domain_conf.c: allocate memory to def->redirdevs in virDomainDefParseXML such as VIR_ALLOC_N(def->redirdevs, n), however, virDomainDefFree(def) hasn't released these memory. * Detected in valgrind run: ==19820== 209 (16 direct, 193 indirect) bytes in 1 blocks are definitely lost in loss record 25 of 26 ==19820== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==19820== by 0x4A13AF: virAllocN (memory.c:129) ==19820== by 0x4D4A0E: virDomainDefParseXML (domain_conf.c:7258) ==19820== by 0x4D4C93: virDomainDefParseNode (domain_conf.c:7512) ==19820== by 0x4D562F: virDomainDefParse (domain_conf.c:7465) ==19820== by 0x415863: testCompareXMLToXMLFiles (qemuxml2xmltest.c:35) ==19820== by 0x415982: testCompareXMLToXMLHelper (qemuxml2xmltest.c:80) ==19820== by 0x416D31: virtTestRun (testutils.c:140) ==19820== by 0x415604: mymain (qemuxml2xmltest.c:192) ==19820== by 0x416437: virtTestMain (testutils.c:689) ==19820== by 0x3CA7A1ECDC: (below main) (in /lib64/libc-2.12.so) ==19820== ==19820== LEAK SUMMARY: ==19820== definitely lost: 16 bytes in 1 blocks ==19820== indirectly lost: 193 bytes in 5 blocks ==19820== possibly lost: 0 bytes in 0 blocks ==19820== still reachable: 1,054 bytes in 21 blocks * How to reproduce? % valgrind -v --leak-check=full ./tests/qemuxml2xmltest 2011-09-16 Peter Krempa build: storage: Macro 'MKFS' is undefined on some platforms. Mac OS X 10.6. Snow Leopard and probably other do not provide a mkfs command to create filesystems. Macro MKFS then remained undefined and did not provide any substitute, so that build failed on a missing argument. build: storage: Conditionaly compiled structure caused build fail on OSX Struct virStoragePoolProbeResult was compiled in conditionaly, but virStorageBackendFileSystemProbe used it unconditionaly. This patch exempts the struct from conditional include. Avoid using "devname" as an identifier. /usr/lib/stdlib.h in Mac OS X and probably also in BSD's exports this symbol :( 2011-09-16 Peter Krempa doc: Add statment about permissions needed to do a core dump Documentation did not specify, that some permissions are required on target path for coredump for the user running the hypervisor. Diff to v1: - reword statements 2011-09-16 Eric Blake snapshot: tweak snapshot-create-as diskspec docs With this patch, it is hopefully a bit more obvious that for snapshot-create-as, a literal '--diskspec' is mandatory if name or description was omitted, but optional if all earlier options were provided. These all denote two diskspecs and a description: virsh snapshot-create-as dom name desc vda vdb virsh snapshot-create-as dom name desc --diskspec vda --diskspec vdb virsh snapshot-create-as dom name desc --diskspec vda vdb virsh snapshot-create-as dom name desc vda --diskspec vdb virsh snapshot-create-as dom --diskspec vda --diskspec vdb name desc This gives two diskspecs but no description: virsh snapshot-create-as dom name --diskspec vda --diskspec vdb And this treats 'vda' as the description, with only one diskspec: virsh snapshot-create-as dom name vda vdb The help output now shows: snapshot-create-as [] [] [--print-xml] [--no-metadata] [--halt] [--disk-only] [[--diskspec] ]... I also checked the help output for echo and send-key, which are two other variants of argv commands. * tools/virsh.pod (snapshot-create-as): Document when a literal --diskspec must preceed a diskspec argument. * tools/virsh.c (vshCmddefHelp): Update help output for argv when naming the option is useful. (vshCmddefGetData): Fix logic on when argv was seen. * tests/virsh-optparse: Add tests to avoid regressions. 2011-09-15 Cole Robinson python: Fix libvirt.py generation to include virterror info Recent generator refactoring broke libvirt.py. With this patch, libvirt.py is generated exactly the same as before offending commit 9eba0d25. 2011-09-15 Eric Blake qemu_api: doc improvements The new doc text had a few readability issues. Also, the monitor command text copied a bit too much from the attach case. * src/libvirt-qemu.c (virDomainQemuMonitorCommand) (virDomainQemuAttach): Fix typos and grammar. 2011-09-15 Peter Krempa doc: virsh: Fix command name in man page Fix cut&paste error having command named domif-setlink instead of domif-getlink. 2011-09-15 Alex Jia sanlock: add missing test command in virt-sanlock-cleanup.in * tools/virt-sanlock-cleanup.in: fix missing test command when judging second condition. https://bugzilla.redhat.com/show_bug.cgi?id=738534 2011-09-15 Michal Privoznik spec: Require augeas for sanlock Even though we BuildRequire augeas in some cases, we need to require it even after if we build with sanlock. virt-sanlock-cleanup use it. 2011-09-15 Jim Fehlig Fix build after commit 829bce17 Pushing under build-breaker rule. 2011-09-15 Jim Fehlig Use max bandwidth from qemuDomainObjPrivate struct when migrating Adjust qemuMigrationRun() to use migMaxBandwidth in qemuDomainObjPrivate structure when setting qemu migration speed. Caller-specified 'resource' parameter overrides migMaxBandwidth. 2011-09-15 Jim Fehlig Set qemu migration speed unlimited when migrating to file The qemu migration speed default is 32MiB/s as defined in migration.c /* Migration speed throttling */ static int64_t max_throttle = (32 << 20); There's no need to throttle migration when targeting a file, so set migration speed to unlimited prior to migration, and restore to libvirt default value after migration. Default units is MB for migrate_set_speed monitor command, so (INT64_MAX / (1024 * 1024)) is used for unlimited migration speed. Tested with both json and text monitors. 2011-09-15 Jim Fehlig Save migration speed in qemuDomainMigrateSetMaxSpeed Now that migration speed is stored in qemuDomainObjPrivate structure, save the new value when invoking qemuDomainMigrateSetMaxSpeed(). Allow setting migration speed on inactive domain too. 2011-09-15 Jim Fehlig Impl virDomainMigrateGetMaxSpeed in qemu driver Store max migration bandwidth in qemuDomainObjPrivate struct The maximum bandwidth that can be consumed when migrating a domain is better classified as an operational vs configuration parameter of the dommain. As such, store this parameter in qemuDomainObjPrivate structure. 2011-09-14 Marc-André Lureau conf: add missing break in virDomainAuditRedirdev Also initialize to NULL a few variables that might get free before being set. 2011-09-14 Eric Blake network: add missing exports Commit c246b025 added new functions, but forgot to export them, resulting in a build failure when using modules. * src/libvirt_private.syms (network.h): Export new functions. 2011-09-14 Eric Blake snapshot: ABI stability must include memory sizing Commit 973fcd8f introduced the ability for qemu to reject snapshot reversion on an ABI incompatibility; but the very example that was first proposed on-list[1] as a demonstration of an ABI incompatibility, namely that of changing the max memory allocation, was not being checked for, resulting in a cryptic failure when running with larger max mem than what the snapshot was created with: error: operation failed: Error -22 while loading VM state This commit merely protects the three variables within mem that are referenced by qemu_command.c, rather than all 7 (the other 4 variables affect cgroup handling, but as far as I can tell, have no visible effect to the qemu guest). This also affects migration and save file handling, which are other places where we perform ABI compatibility checks. [1] https://www.redhat.com/archives/libvir-list/2010-December/msg00331.html * src/conf/domain_conf.c (virDomainDefCheckABIStability): Add memory sizing checks. 2011-09-14 Eric Blake maint: ignore generated files * .gitignore: Exempt recently added generated files. 2011-09-14 Peter Krempa xml: Clean up rest of virtual XML document names for XML strings Commit 498d783 cleans up some of virtual file names for parsing strings in memory. This patch cleans up (hopefuly) the rest forgotten by the first patch. This patch also changes all of the previously modified "filenames" to valid URI's replacing spaces for underscores. Changes to v1: - Replace all spaces for underscores, so that the strings form valid URI's - Replace spaces in places changed by commit 498d783 2011-09-14 Daniel Veillard Remove two references to files not generated This was breaking "make dist" 2011-09-14 Peter Krempa make: Fix 'make syntax-check' failing. Commit 2a0d75e5 added file python/libvirt-qemu-override.c that contains code that does not pass "make syntax-check". This patch adds an exception for this file and the check. prohibit_always_true_header_tests python/libvirt-qemu-override.c:17:#undef HAVE_PTHREAD_H maint.mk: do not test the above HAVE_
_H symbol(s); with the corresponding gnulib module, they are always true make: *** [sc_prohibit_always_true_header_tests] Error 1 2011-09-14 Osier Yang qemu_api: Update libvirt spec file qemu_api: Update Makefile to generate libvirtmod_qemu lib qemu_api: Update Py binding generator to generate files for QEMU APIs It will generate: libvirt-qemu.py libvirt-qemu.h libvirt-qemu.c libvirt-qemu-export.c qemu_api: Add override XML and C files for QEMU APIs There is only one function (virDomainQemuMonitorCommand) need to be hand-craft. qemu_api: Add comments for API virDomainQemuMonitorCommand And fix argument @pid's type of virDomainQemuAttach. qemu_api: Update Makefile for subdir docs qemu_api: Modify apibuild.py to generate docs for QEMU APIs The generated docs are: libvirt-qemu-api.xml, libvirt-qemu-refs.xml 2011-09-14 Peter Krempa virsh: Use old API if remote libvirtd does not support new Commit ffe28ab74b821c916ec4ba8efb5c992454e4bd24 introduced regression while communicating with older libvirtd command 'domblkstat' used the new API and did not check for VIR_ERR_RPC error code signalling the remote server does not support this API and did not fall back to older API. Thereafter 'domblkstat' ended with "error: unknown procedure: 243". 2011-09-13 Guannan Ren snapshot: fix double free of qemuImgBinary Regression introduced in commit 3881a470, due to an improper rebase of a cleanup written beforehand but only applied after a rebased of a refactoring that created a new function in commit 25fb3ef. Also avoids passing NULL to printf %s. * src/qemu/qemu_driver.c: In qemuDomainSnapshotForEachQcow2() it free up the memory of qemu_driver->qemuImgBinary in the cleanup tag which leads to the garbage value of qemuImgBinary in qemu_driver struct and libvirtd crash when running "virsh snapshot-create" command a second time. 2011-09-13 Daniel Veillard Do not log invalid operations in libvirtd logs This is a bit painful for example when starting virt-manager it tends to clutter libvirtd.log with invalid operation on cpu pinning for defined but not running domains. A priori those kind of errors don't indicate an error when executing the command but on a precondition for running the API, and honnestly while the application should report it, logging it as an error in libvirtd.log is not really useful, Related bug: https://bugzilla.redhat.com/show_bug.cgi?id=590807 * daemon/libvirtd.c: extend daemonErrorLogFilter() to filter out errors of type VIR_ERR_OPERATION_INVALID 2011-09-09 Michal Privoznik virnetsocket: Pass KRB5CCNAME env variable So we can allow GSSAPI authentication for ssh. selinux: Detect virt_use_nfs boolean set If we fail setting label on a file and this file is on NFS share, it is wise to advise user to set virt_use_nfs selinux boolean variable. 2011-09-09 Philipp Hahn Fix URL-escaping for domainDefine '+' in strings get translated to ' ' when editing domains. While xenDaemonDomainCreateXML() did URL-escape the sexpr, xenDaemonDomainDefineXML() did not. Remove the explicit urlencode() in xenDaemonDomainCreateXML() and add the direct encoding calls to xend_op_ext() because it calls xend_post() which uses "Content-Type: application/x-www-form-urlencoded". According to this requires all parameters to be url-encoded as specified in rfc1738. Notice: virBufferAsprintf(..., "%s=%s", ...) is again replaced by three calls to virBufferURIEncodeString() and virBufferAddChar() because '=' is a "reserved" character, which would get escaped by virBufferURIEncodeString(), which - by the way - escapes anything not c_isalnum(). 2011-09-08 Peter Krempa xml: Change virtual file names of xml documents parsed in memory While parsing XML strings from memory, the previous convention in libvirt was to set the virtual file name to "domain.xml" or something similar. This could potentialy trick the user into looking for a file named domain.xml on the disk in an attempt to fix the error. This patch changes these filenames to something that can't be as easily confused for a valid filename. Examples of error messages: --------------------------- Error while loading file from disk: 15:07:59.015: 527: error : catchXMLError:709 : /path/to/domain.xml:1: StartTag: invalid element name < --------------------^ Error while parsing definition in memory: 15:08:43.581: 525: error : catchXMLError:709 : (domain definition):2: error parsing attribute name vm1 --^ 2011-09-08 Eric Blake snapshot: fix regression with system checkpoints Regression introduced in commit d6f6b2d194c. Running 'virsh snapshot-create dom' would mistakenly report that disks can only be specified for disk snapshots. * src/conf/domain_conf.c (virDomainSnapshotDefParseString): Only give error about no disk support when was found. 2011-09-08 Eric Blake maint: update to latest gnulib * .gnulib: Update to latest. 2011-09-08 Eric Blake snapshot: use new API for less work This has the added benefit of making 'snapshot-create dom --no-metadata' now able to tell you the name of the just-generated snapshot. * tools/virsh.c (vshSnapshotCreate, cmdSnapshotCurrent): Don't get XML just for name. 2011-09-08 Eric Blake snapshot: new APIs for inspecting snapshot object These functions access internals of the opaque object, and do not need any rpc counterpart. It could be argued that we should have provided these when snapshot objects were first introduced, since all the other vir*Ptr objects have at least a GetName accessor. * include/libvirt/libvirt.h.in (virDomainSnapshotGetName) (virDomainSnapshotGetDomain, virDomainSnapshotGetConnect): Declare. * src/libvirt.c (virDomainSnapshotGetName) (virDomainSnapshotGetDomain, virDomainSnapshotGetConnect): New functions. * src/libvirt_public.syms: Export them. 2011-09-08 Soren Hansen Don't treat pci_system_init failure as fatal if no PCI bus is present Xen PV domU's have no PCI bus. node_device_udev.c calls pci_system_init which looks for /sys/bus/pci. If it does not find /sys/bus/pci (which it won't in a Xen PV domU) it returns unsuccesfully (ENOENT), which libvirt considers fatal. This makes libvirt unusable in this environment, even though there are plenty of valid virtualisation options that work there (LXC, UML, and QEmu spring to mind) https://bugzilla.redhat.com/show_bug.cgi?id=709471 2011-09-08 Alex Jia doc: fix incorrect option in send-key * tools/virsh.pod: fix a option typo of send-key section. virsh: fix typo in opts_send_key * tools/virsh.c: millseconds should be milliseconds in opts_send_key. 2011-09-08 Alex Jia rpc: avoid memory leak on virNetTLSContextValidCertificate * src/rpc/virnettlscontext.c: fix memory leak on virNetTLSContextValidCertificate. * Detected in valgrind run: ==25667== ==25667== 6,085 (44 direct, 6,041 indirect) bytes in 1 blocks are definitely lost in loss record 326 of 351 ==25667== at 0x4005447: calloc (vg_replace_malloc.c:467) ==25667== by 0x4F2791F3: _asn1_add_node_only (structure.c:53) ==25667== by 0x4F27997A: _asn1_copy_structure3 (structure.c:421) ==25667== by 0x4F276A50: _asn1_append_sequence_set (element.c:144) ==25667== by 0x4F2743FF: asn1_der_decoding (decoding.c:1194) ==25667== by 0x4F22B9CC: gnutls_x509_crt_import (x509.c:229) ==25667== by 0x805274B: virNetTLSContextCheckCertificate (virnettlscontext.c:1009) ==25667== by 0x804DE32: testTLSSessionInit (virnettlscontexttest.c:693) ==25667== by 0x804F14D: virtTestRun (testutils.c:140) ==25667== ==25667== 23,188 (88 direct, 23,100 indirect) bytes in 11 blocks are definitely lost in loss record 346 of 351 ==25667== at 0x4005447: calloc (vg_replace_malloc.c:467) ==25667== by 0x4F22B841: gnutls_x509_crt_init (x509.c:50) ==25667== by 0x805272B: virNetTLSContextCheckCertificate (virnettlscontext.c:1003) ==25667== by 0x804DDD1: testTLSSessionInit (virnettlscontexttest.c:673) ==25667== by 0x804F14D: virtTestRun (testutils.c:140) * How to reproduce? % cd libvirt && ./configure && make && make -C tests valgrind or % valgrind -v --leak-check=full ./tests/virnettlscontexttest 2011-09-08 Alex Jia tests: avoid memory leak on testTLSSessionInit * tests/virnettlscontexttest: fix memory leak on virnettlscontext test case. * Detected in valgrind run: ==25667== ==25667== 86,651 (34,680 direct, 51,971 indirect) bytes in 10 blocks are definitely lost in loss record 350 of 351 ==25667== at 0x4005447: calloc (vg_replace_malloc.c:467) ==25667== by 0x4F1F515D: gnutls_init (gnutls_state.c:270) ==25667== by 0x8053432: virNetTLSSessionNew (virnettlscontext.c:1181) ==25667== by 0x804DD24: testTLSSessionInit (virnettlscontexttest.c:624) ==25667== by 0x804F14D: virtTestRun (testutils.c:140) ==25667== ==25667== 100,578 (38,148 direct, 62,430 indirect) bytes in 11 blocks are definitely lost in loss record 351 of 351 ==25667== at 0x4005447: calloc (vg_replace_malloc.c:467) ==25667== by 0x4F1F515D: gnutls_init (gnutls_state.c:270) ==25667== by 0x8053432: virNetTLSSessionNew (virnettlscontext.c:1181) ==25667== by 0x804DD3C: testTLSSessionInit (virnettlscontexttest.c:625) ==25667== by 0x804F14D: virtTestRun (testutils.c:140) * How to reproduce? % cd libvirt && ./configure && make && make -C tests valgrind or % valgrind -v --leak-check=full ./tests/virnettlscontexttest 2011-09-08 Alex Jia libxl: avoid a dereference of a null pointer Variable 'l_disk' initialized to a null pointer value, control jumps to 'case VIR_DOMAIN_DISK_DEVICE_DISK and then taking false branch, Within the expansion of the macro 'libxlError': Field access results in a dereference of a null pointer (loaded from variable 'l_disk'). * src/libxl/libxl_driver.c: Field access results in a dereference of a null pointer (loaded from variable 'l_disk') 2011-09-08 Eric Blake blockinfo: fix qemu regression in handling disk name Regression introduced in commit 89b6284fd, due to an incorrect conversion to the new means of converting disk names back to the correct object. * src/qemu/qemu_driver.c (qemuDomainGetBlockInfo): Avoid NULL deref. 2011-09-08 Matthias Bolte esx: Fix managed object lookup with optional occurrence Exit early if managed object is not found, instead of dereferencing a NULL pointer and triggering a segfault. 2011-09-07 Michal Privoznik Threadpool: Initialize new dynamic workers Although we were initializing worker threads during pool creating, we missed this during virThreadPoolSendJob. This bug led to segmenation fault as worker thread free() given argument. 2011-09-06 Matthias Bolte mingw: Don't use interface as an identifier Because it's a define used in MSCOM and its usage as identifier results in a compile error. 2011-09-06 Peter Krempa link-state: virsh: Add wrapper commands for changing link state Two new commands are added to virsh that wrap usage of virDomainUpdateDeviceFlags for changing link state of domain's network interfaces. These wrappers extract network devices's xml configuration and modify the link state for easy manipulation from an user's perspective. - domif-setlink - set link state of a domains virtual network interface - domif-getlink - get link state * tools/virsh.c - Add functionality to virsh * tools/virsh.pod - Manpage documentation 2011-09-06 Peter Krempa link-state: qemu: Add net intf modification to virUpdateDeviceFlags This patch enables modifying network device configuration using the virUpdateDeviceFlags API method. Matching of devices is accomplished using MAC addresses. While updating live configuration of a running domain, the user is allowed only to change link state of the interface. Additional modifications may be added later. For now the code checks for unsupported changes and thereafter changes the link state, if applicable. When updating persistent configuration of guest's network interface the whole configuration (except for the MAC address) may be modified and is stored for the next startup. * src/qemu/qemu_driver.c - Add dispatching of virUpdateDevice for network devices update (live/config) * src/qemu/qemu_hotplug.c - add setting of initial link state on live device addition - add function to change network device configuration. By now it supports only changing of link state * src/qemu/qemu_hotplug.h - Headers to above functions * src/qemu/qemu_process.c - set link states before virtual machine start. Qemu does not support setting of this on the command line. 2011-09-06 Peter Krempa link-state: qemu: Add monitor handling for link state modification This patch adds handlers for modification of guest's interface link state. Both HMP and QMP commands are supported, but as the link state functionality is from the beginning supported in QMP the HMP code will probably never be used. 2011-09-06 Peter Krempa link-state: conf: Add element to XML for controling link state A new element is introduced to XML that allows to control state of virtual network interfaces in hypervisors. Live modification of the link state allows networking tools propagate topology changes to guest OS or testing of scenarios in complex (virtual) networks. This patch adds elements to XML grammars and parsing and generating code. 2011-09-06 Peter Krempa link-state: util: Add equality comparison functions for structures This patch adds functions to compare structures containing network device configuration for equality. They serve for the purpose of disallowing unsupported changes to live network devices. 2011-09-06 Peter Krempa XML: Improve XML parsing error messages This patch modifies error handling function for the XML parser provided by libxml2. Originaly only a line number and error message were logged. With this new error handler function, the user is provided with a more complex description of the parsing error. Context of the error is printed in libXML2 style and filename of the file, that caused the error is printed. Example of an parse error: 13:41:36.262: 16032: error : catchXMLError:706 : /etc/libvirt/qemu/rh_bad.xml:58: Opening and ending tag mismatch: name line 2 and domain ---------^ Context of the error gives the user hints that may help to quickly locate a corrupt xml file. fixes BZs: ---------- Bug 708735 - [RFE] Show column and line on XML parsing error https://bugzilla.redhat.com/show_bug.cgi?id=708735 Bug 726771 - libvirt does not specify problem file if persistent xml is invalid https://bugzilla.redhat.com/show_bug.cgi?id=726771 2011-09-06 Marc-André Lureau redirdev: allows to specify device address It is important to be able to attach USB redirected devices to a particular controller (one that supports USB2 for instance). Without this patch, only the default bus was used.
2011-09-06 Daniel Veillard latency: fix make check for remote protocol structs and numbers 2011-09-06 Osier Yang latency: Update virsh command domblkstat to use new API The modified function fallbacks to use virDomainBlockStats if virDomainBlockStatsFlags is not supported by the hypervisor driver. If the new API is supported, it will be invoked instead of the old API. latency: Expose the new API for Python binding latency: Implemente internal API for qemu driver 2011-09-06 Osier Yang latency: Update monitor functions for new latency fields The mainly changes are: 1) Update qemuMonitorGetBlockStatsInfo and it's children (Text/JSON) functions to return the value of new latency fields. 2) Add new function qemuMonitorGetBlockStatsParamsNumber, which is to count how many parameters the underlying QEMU supports. 3) Update virDomainBlockStats in src/qemu/qemu_driver.c to be compatible with the changes by 1). 2011-09-06 Osier Yang latency: Wire up the remote protocol latency: Implemente the public API latency: Define the internal driver callback latency: Define new public API and structure 2011-09-05 Michal Privoznik qemu: Deal with stucked qemu on daemon startup If libvirt daemon gets restarted and there is (at least) one unresponsive qemu, the startup procedure hangs up. This patch creates one thread per vm in which we try to reconnect to monitor. Therefore, blocking in one thread will not affect other APIs. qemu: Introduce job queue size limit This patch creates an optional BeginJob queue size limit. When active, all other attempts above level will fail. To set this feature assign desired value to max_queued variable in qemu.conf. Setting it to 0 turns it off. 2011-09-05 Michal Privoznik daemon: Create priority workers pool This patch annotates APIs with low or high priority. In low set MUST be all APIs which might eventually access monitor (and thus block indefinitely). Other APIs may be marked as high priority. However, some must be (e.g. domainDestroy). For high priority calls (HPC), there are some high priority workers (HPW) created in the pool. HPW can execute only HPC, although normal worker can process any call regardless priority. Therefore, only those APIs which are guaranteed to end in reasonable small amount of time can be marked as HPC. The size of this HPC pool is static, because HPC are expected to end quickly, therefore jobs assigned to this pool will be served quickly. It can be configured in libvirtd.conf via prio_workers variable. Default is set to 5. To mark API with low or high priority, append priority:{low|high} to it's comment in src/remote/remote_protocol.x. This is similar to autogen|skipgen. If not marked, the generator assumes low as default. 2011-09-05 Eric Blake snapshot: use SELinux and lock manager with external snapshots With this, it is now possible to create external snapshots even when SELinux is enforcing, and to protect the new file with a lock manager. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateSingleDiskActive): Create and register new file with proper permissions and locks. (qemuDomainSnapshotCreateDiskActive): Update caller. 2011-09-05 Eric Blake snapshot: wire up live qemu disk snapshots Lots of earlier patches led up to this point - the qemu snapshot_blkdev monitor command can now be controlled by libvirt! Well, insofar as SELinux doesn't prevent qemu from open(O_CREAT) on the files. There's still some followup work before things work with SELinux enforcing, but this patch is big enough to post now. There's still room for other improvements, too (for example, taking a disk snapshot of an inactive domain, by using qemu-img for both internal and external snapshots; wiring up delete and revert control, including additional flags from my RFC; supporting active QED disk snapshots; supporting per-storage-volume snapshots such as LVM or btrfs snapshots; etc.). But this patch is the one that proves the new XML works! * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Wire in active disk snapshots. (qemuDomainSnapshotDiskPrepare) (qemuDomainSnapshotCreateDiskActive) (qemuDomainSnapshotCreateSingleDiskActive): New functions. 2011-09-05 Eric Blake snapshot: wire up new qemu monitor command No one uses this yet, but it will be important once virDomainSnapshotCreateXML learns a VIR_DOMAIN_SNAPSHOT_DISK_ONLY flag, and the xml allows passing in the new file names. * src/qemu/qemu_monitor.h (qemuMonitorDiskSnapshot): New prototype. * src/qemu/qemu_monitor_text.h (qemuMonitorTextDiskSnapshot): Likewise. * src/qemu/qemu_monitor_json.h (qemuMonitorJSONDiskSnapshot): Likewise. * src/qemu/qemu_monitor.c (qemuMonitorDiskSnapshot): New function. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot): Likewise. 2011-09-05 Eric Blake snapshot: make it possible to audit external snapshot Snapshots alter the set of disk image files opened by qemu, so they must be audited. But they don't involve a full disk definition structure, just the new filename. Make the next patch easier by refactoring the audit routines to just operate on file name. * src/conf/domain_audit.h (virDomainAuditDisk): Update prototype. * src/conf/domain_audit.c (virDomainAuditDisk): Act on strings, not definition structures. (virDomainAuditStart): Update caller. * src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia) (qemuDomainAttachPciDiskDevice, qemuDomainAttachSCSIDisk) (qemuDomainAttachUsbMassstorageDevice) (qemuDomainDetachPciDiskDevice, qemuDomainDetachDiskDevice): Likewise. 2011-09-05 Eric Blake snapshot: reject unimplemented disk snapshot features My RFC for snapshot support [1] proposes several rules for when it is safe to delete or revert to an external snapshot, predicated on the existence of new API flags. These will be incrementally added in future patches, but until then, blindly mishandling a disk snapshot risks corrupting internal state, so it is better to outright reject the attempts until the other pieces are in place, thus incrementally relaxing the restrictions added in this patch. [1] https://www.redhat.com/archives/libvir-list/2011-August/msg00361.html * src/qemu/qemu_driver.c (qemuDomainSnapshotCountExternal): New function. (qemuDomainUndefineFlags, qemuDomainSnapshotDelete): Use it to add safety valve. (qemuDomainRevertToSnapshot, qemuDomainSnapshotCreateXML): Add safety valve. 2011-09-05 Eric Blake snapshot: wire up disk-only flag to snapshot-create Expose the disk-only flag through virsh. Additionally, make virsh snapshot-create-as take an arbitrary number of diskspecs, which can be used to build up the xml for . * tools/virsh.c (cmdSnapshotCreate): Add --disk-only. (cmdSnapshotCreateAs): Likewise, and add argv diskspec. (vshParseSnapshotDiskspec): New helper function. (vshCmddefGetOption): Allow naming of argv field. * tools/virsh.pod (snapshot-create, snapshot-create-as): Document them. * tests/virsh-optparse: Test snapshot-create-as parsing. 2011-09-05 Eric Blake snapshot: add flag for requesting disk snapshot Prior to this patch, / was ignored. This changes it to be an error unless an explicit disk snapshot is requested (a future patch may relax things if it turns out to be useful to have a specification alongside a system checkpoint). * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY): New flag. * src/libvirt.c (virDomainSnapshotCreateXML): Document it. * src/esx/esx_driver.c (esxDomainSnapshotCreateXML): Disk snapshots not supported yet. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotCreateXML): Likewise. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Likewise. 2011-09-05 Eric Blake snapshot: add virsh domblklist command This adds a convenience function to virsh that parses out block information from the domain xml, making it much easier to see what strings can be used in all other contexts that demand a specific block name, especially when given the previous patch that allows using either target or unique source name. As an example on a domain with one disk and an empty cdrom drive: Target Source ------------------------------------------- vda /var/lib/libvirt/images/fedora_12.img hdc - * tools/virsh.c (cmdDomblklist): New function. * tools/virsh.pod (domblklist): Document it. 2011-09-05 Eric Blake snapshot: also support disks by path I got confused when 'virsh domblkinfo dom disk' required the path to a disk (which can be ambiguous, since a single file can back multiple disks), rather than the unambiguous target device name that I was using in disk snapshots. So, in true developer fashion, I went for the best of both worlds - all interfaces that operate on a disk (aka block) now accept either the target name or the unambiguous path to the backing file used by the disk. * src/conf/domain_conf.h (virDomainDiskIndexByName): Add parameter. (virDomainDiskPathByName): New prototype. * src/libvirt_private.syms (domain_conf.h): Export it. * src/conf/domain_conf.c (virDomainDiskIndexByName): Also allow searching by path, and decide whether ambiguity is okay. (virDomainDiskPathByName): New function. (virDomainDiskRemoveByName, virDomainSnapshotAlignDisks): Update callers. * src/qemu/qemu_driver.c (qemudDomainBlockPeek) (qemuDomainAttachDeviceConfig, qemuDomainUpdateDeviceConfig) (qemuDomainGetBlockInfo, qemuDiskPathToAlias): Likewise. * src/qemu/qemu_process.c (qemuProcessFindDomainDiskByPath): Likewise. * src/libxl/libxl_driver.c (libxlDomainAttachDeviceDiskLive) (libxlDomainDetachDeviceDiskLive, libxlDomainAttachDeviceConfig) (libxlDomainUpdateDeviceConfig): Likewise. * src/uml/uml_driver.c (umlDomainBlockPeek): Likewise. * src/xen/xend_internal.c (xenDaemonDomainBlockPeek): Likewise. * docs/formatsnapshot.html.in: Update documentation. * tools/virsh.pod (domblkstat, domblkinfo): Likewise. * docs/schemas/domaincommon.rng (diskTarget): Tighten pattern on disk targets. * docs/schemas/domainsnapshot.rng (disksnapshot): Update to match. * tests/domainsnapshotxml2xmlin/disk_snapshot.xml: Update test. 2011-09-05 Eric Blake snapshot: add to snapshot xml Adds an optional element to , which will be used to give user control over external snapshot filenames on input, and specify generated filenames on output. For now, no driver accepts this element; that will come later. ... ... * src/conf/domain_conf.h (_virDomainSnapshotDiskDef): New type. (_virDomainSnapshotDef): Add new elements. (virDomainSnapshotAlignDisks): New prototype. * src/conf/domain_conf.c (virDomainSnapshotDiskDefClear) (virDomainSnapshotDiskDefParseXML, disksorter) (virDomainSnapshotAlignDisks): New functions. (virDomainSnapshotDefParseString): Parse new fields. (virDomainSnapshotDefFree): Clean them up. (virDomainSnapshotDefFormat): Output them. * src/libvirt_private.syms (domain_conf.h): Export new function. * docs/schemas/domainsnapshot.rng (domainsnapshot, disksnapshot): Add more xml. * docs/formatsnapshot.html.in: Document it. * tests/domainsnapshotxml2xmlin/disk_snapshot.xml: New test. * tests/domainsnapshotxml2xmlout/disk_snapshot.xml: Update. 2011-09-05 Eric Blake snapshot: support extra state in snapshots In order to distinguish disk snapshots from system checkpoints, a new state value that is only valid for snapshots is helpful. * include/libvirt/libvirt.h.in (VIR_DOMAIN_LAST): New placeholder. * src/conf/domain_conf.h (virDomainSnapshotState): New enum mapping. (VIR_DOMAIN_DISK_SNAPSHOT): New internal enum value. * src/conf/domain_conf.c (virDomainState): Use placeholder. (virDomainSnapshotState): Extend mapping by one for use in snapshot. (virDomainSnapshotDefParseString, virDomainSnapshotDefFormat): Handle new state. (virDomainObjSetState, virDomainStateReasonToString) (virDomainStateReasonFromString): Avoid compiler warnings. * tools/virsh.c (vshDomainState, vshDomainStateReasonToString): Likewise. * src/libvirt_private.syms (domain_conf.h): Export new functions. * docs/schemas/domainsnapshot.rng: Tighten state definition. * docs/formatsnapshot.html.in: Document it. * tests/domainsnapshotxml2xmlout/disk_snapshot.xml: New test. 2011-09-05 Eric Blake snapshot: expose halt-after-creation in virsh Easy enough to emulate even with older servers. * tools/virsh.c (cmdSnapshotCreate, cmdSnapshotCreateAs): Add --halt flag. (vshSnapshotCreate): Emulate halt when flag is unsupported. * tools/virsh.pod (snapshot-create, snapshot-create-as): Document it. 2011-09-05 Eric Blake snapshot: allow halting after snapshot Since a snapshot is fully recoverable, it is useful to have a snapshot as a means of hibernating a guest, then reverting to the snapshot to wake the guest up. This mode of usage is similar to 'virsh save/virsh restore', except that virsh save uses an external file while virsh snapshot keeps the vm state internal to a qcow2 file. However, it only works on persistent domains. In the usage pattern of snapshot/revert for hibernating a guest, there is no need to keep the guest running between the two points in time, especially since that would generate runtime state that would just be discarded. Add a flag to make it possible to stop the domain after the snapshot has completed. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_CREATE_HALT): New flag. * src/libvirt.c (virDomainSnapshotCreateXML): Document it. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML) (qemuDomainSnapshotCreateActive): Implement it. 2011-09-05 Eric Blake snapshot: expose new delete flag in virsh It would technically be possible to have virsh compute the list of descendants of a given snapshot, then delete those one at a time. But it's complex, and not worth writing for a first cut at implementing the new flags. * tools/virsh.c (cmdSnapshotDelete): Add --children-only, --metadata. * tools/virsh.pod (snapshot-delete): Document them. 2011-09-05 Eric Blake snapshot: introduce new deletion flag Reverting to a state prior to an external snapshot risks corrupting any other branches in the snapshot hierarchy that were using the snapshot as a read-only backing file. So disk snapshot code will default to preventing reverting to a snapshot that has any children, meaning that deleting just the children of a snapshot becomes a useful operation in preparing that snapshot for being a future reversion target. The code for the new flag is simple - it's one less deletion, plus a tweak to keep the current snapshot correct. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_DELETE_CHILDREN_ONLY): New flag. * src/libvirt.c (virDomainSnapshotDelete): Document it, and enforce mutual exclusion. * src/qemu/qemu_driver.c (qemuDomainSnapshotDelete): Implement it. 2011-09-05 Eric Blake snapshot: reject transient disks where code is not ready The previous patch introduced new config, but if a hypervisor does not support that new config, someone can write XML that does not behave as documented. This prevents some of those cases by explicitly rejecting transient disks for several hypervisors. Disk snapshots will require a new flag to actually affect a snapshot creation, so there's not much to reject there. * src/qemu/qemu_command.c (qemuBuildDriveStr): Reject transient disks for now. * src/libxl/libxl_conf.c (libxlMakeDisk): Likewise. * src/xenxs/xen_sxpr.c (xenFormatSxprDisk): Likewise. * src/xenxs/xen_xm.c (xenFormatXMDisk): Likewise. 2011-09-05 Eric Blake snapshot: additions to domain xml for disks As discussed here: https://www.redhat.com/archives/libvir-list/2011-August/msg00361.html https://www.redhat.com/archives/libvir-list/2011-August/msg00552.html Adds snapshot attribute and transient sub-element: ... * docs/schemas/domaincommon.rng (snapshot): New define. (disk): Add snapshot and persistent attributes. * docs/formatdomain.html.in: Document them. * src/conf/domain_conf.h (virDomainDiskSnapshot): New enum. (_virDomainDiskDef): New fields. * tests/qemuxml2argvdata/qemuxml2argv-disk-transient.xml: New test of rng, no args counterpart until qemu support is complete. * tests/qemuxml2argvdata/qemuxml2argv-disk-snapshot.args: New file, snapshot attribute does not affect args. * tests/qemuxml2argvdata/qemuxml2argv-disk-snapshot.xml: Likewise. * tests/qemuxml2argvtest.c (mymain): Run new test. 2011-09-05 Lei Li Check for source conflicts in storage pools Fix bug #611823 storage driver should prohibit pools with duplicate underlying storage. Add internal API virStoragePoolSourceFindDuplicate() to do uniqueness check based on source location infomation for pool type. * AUTHORS: add Lei Li 2011-09-05 Marc-André Lureau Add a usb1 & usb2 qemuxml2argv test Default USB device is on slot 1 function 2 Fix qemuAssignDevicePCISlots() and the associated regression tests PIIX3 USB controller is on function 2 Current code reserves slot 1 function 2 even if there is a user defined PIIX3 USB controller there. 2011-09-05 Philipp Hahn Fix localtime handling for Xen-PV domains At least Xen-3.4.3 translates the /vm/localtime SXPR value to /domain/platform/localtime and /domain/image/{linux,hvm}/localtime when the domain is defined. When reading back that information libvirt only handles HVM domains, but not PV domains: This results in libvirtd always returning while Xend used (localtime 1). For PV domains use /domain/image/linux/localtime. 2011-09-03 Eric Blake snapshot: store qemu domain details in xml When reverting to a snapshot, the inactive domain configuration has to be rolled back to what it was at the time of the snapshot. Additionally, if the VM is active and the snapshot was active, this now adds a failure if the two configurations are ABI incompatible, rather than risking qemu confusion. A future patch will add a VIR_DOMAIN_SNAPSHOT_FORCE flag, which will be required for two risky code paths - reverting to an older snapshot that lacked full domain information, and reverting from running to a live snapshot that requires starting a new qemu process. Any reverting that stops a running vm is also a form of data loss (discarding the current running state to go back in time), but as that is what reversion usually implies, it is probably not worth requiring a force flag. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Copy out domain. (qemuDomainSnapshotCreateXML, qemuDomainRevertToSnapshot): Perform ABI compatibility checks. 2011-09-03 Eric Blake snapshot: update rng to support full domain in xml This patch will probably cause merge conflicts to those trying to do backports. The end goal is simple - domaincommon.rng should be the state of domain.rng pre-patch, with a few lines tweaked in the header, while domain.rng post-patch is now just a shell that includes domaincommon.rng and sets the . * docs/schemas/domain.rng: Move guts... * docs/schemas/domaincommon.rng: ...to new file. * docs/schemas/domainsnapshot.rng: Allow new xml. * docs/schemas/Makefile.am (schema_DATA): Distribute new file. * tests/domainsnapshotxml2xmlout/full_domain.xml: New test. * libvirt.spec.in (%files client): Ship new file. Sort lines. * mingw32-libvirt.spec.in: Likewise. 2011-09-03 Eric Blake snapshot: correctly escape generated xml Commit 69278878 fixed one direction of arbitrarily-named snapshots, but not the round trip path. While auditing domain_conf, I found a couple other instances that weren't escaping arbitrary strings. * src/conf/domain_conf.c (virDomainFSDefFormat) (virDomainGraphicsListenDefFormat, virDomainSnapshotDefFormat): Escape arbitrary strings. 2011-09-03 Eric Blake snapshot: allow full domain xml in snapshot Just like VM saved state images (virsh save), snapshots MUST track the inactive domain xml to detect any ABI incompatibilities. The indentation is not perfect, but functionality comes before form. Later patches will actually supply a full domain; for now, this wires up the storage to support one, but doesn't ever generate one in dumpxml output. Happily, libvirt.c was already rejecting use of VIR_DOMAIN_XML_SECURE from read-only connections, even though before this patch, there was no information to be secured by the use of that flag. And while we're at it, mark the libvirt snapshot metadata files as internal-use only. * src/libvirt.c (virDomainSnapshotGetXMLDesc): Document flag. * src/conf/domain_conf.h (_virDomainSnapshotDef): Add member. (virDomainSnapshotDefParseString, virDomainSnapshotDefFormat): Update signature. * src/conf/domain_conf.c (virDomainSnapshotDefFree): Clean up. (virDomainSnapshotDefParseString): Optionally parse domain. (virDomainSnapshotDefFormat): Output full domain. * src/esx/esx_driver.c (esxDomainSnapshotCreateXML) (esxDomainSnapshotGetXMLDesc): Update callers. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotCreateXML) (vboxDomainSnapshotGetXMLDesc): Likewise. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML) (qemuDomainSnapshotLoad, qemuDomainSnapshotGetXMLDesc) (qemuDomainSnapshotWriteMetadata): Likewise. * docs/formatsnapshot.html.in: Rework doc example. Based on a patch by Philipp Hahn. 2011-09-03 Eric Blake snapshot: refactor domain xml output Minor semantic change - allow domain xml to be generated in place within a larger buffer, rather than having to go through a temporary string. * src/conf/domain_conf.c (virDomainDefFormatInternal): Add parameter. (virDomainDefFormat, virDomainObjFormat): Update callers. 2011-09-03 Eric Blake snapshot: prevent migration from stranding snapshot data Migration is another case of stranding metadata. And since snapshot metadata is arbitrarily large, there's no way to shoehorn it into the migration cookie of migration v3. This patch consolidates two existing locations for migration validation into one helper function, then enhances that function to also do the new checks. If we could always trust the source to validate migration, then the destination would not have to do anything; but since older servers that did not do checking can migrate to newer destinations, we have to repeat some of the same checks on the destination; meanwhile, we want to detect failures as soon as possible. With migration v2, this means that validation will reject things at Prepare on the destination if the XML exposes the problem, otherwise at Perform on the source; with migration v3, this means that validation will reject things at Begin on the source, or if the source is old and the XML exposes the problem, then at Prepare on the destination. This patch is necessarily over-strict. Once a later patch properly handles auto-cleanup of snapshot metadata on the death of a transient domain, then the only time we actually need snapshots to prevent migration is when using the --undefinesource flag on a persistent source domain. It is possible to recreate snapshot metadata on the destination with VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE and VIR_DOMAIN_SNAPSHOT_CREATE_CURRENT. But for now, that is limited, since if we delete the snapshot metadata prior to migration, then we won't know the name of the current snapshot to pass along; and if we delete the snapshot metadata after migration and use the v3 migration cookie to pass along the name of the current snapshot, then we need a way to bypass the fact that this patch refuses migration with snapshot metadata present. So eventually, we may have to introduce migration protocol v4 that allows feature negotiation and an arbitrary number of handshake exchanges, so as to pass as many rpc calls as needed to transfer all the snapshot xml hierarchy. But all of that is thoughts for the future; for now, the best course of action is to quit early, rather than get into a funky state of stale metadata; then relax restrictions later. * src/qemu/qemu_migration.h (qemuMigrationIsAllowed): Make static. * src/qemu/qemu_migration.c (qemuMigrationIsAllowed): Alter signature, and allow checks for both outgoing and incoming. (qemuMigrationBegin, qemuMigrationPrepareAny) (qemuMigrationPerformJob): Update callers. 2011-09-03 Eric Blake snapshot: support new undefine flags in qemu A nice benefit of deleting all snapshots at undefine time is that you don't have to do any reparenting or subtree identification - since everything goes, this is an O(n) process, whereas using multiple virDomainSnapshotDelete calls would be O(n^2) or worse. But it is only doable for snapshot metadata, where we are in control of the data being deleted; for the actual snapshots, there's too much likelihood of something going wrong, and requiring even more API calls to figure out what failed in the meantime, so callers are better off deleting the snapshot data themselves one snapshot at a time where they can deal with failures as they happen. * src/qemu/qemu_driver.c (qemuDomainUndefineFlags): Honor new flags. 2011-09-03 Eric Blake snapshot: cache qemu-img location As more clients start to want to know this information, doing a PATH stat walk and malloc for every client adds up. We are only caching the location, not the capabilities, so even if qemu-img is updated in the meantime, it will still probably live in the same location. So there is no need to worry about clearing this particular cache. * src/qemu/qemu_conf.h (qemud_driver): Add member. * src/qemu/qemu_driver.c (qemudShutdown): Cleanup. (qemuFindQemuImgBinary): Add an argument, and cache result. (qemuDomainSnapshotForEachQcow2, qemuDomainSnapshotDiscard) (qemuDomainSnapshotCreateInactive, qemuDomainSnapshotRevertInactive) (qemuDomainSnapshotCreateXML, qemuDomainRevertToSnapshot): Update callers. 2011-09-03 Eric Blake snapshot: refactor some qemu code Prepare for code sharing. No semantic change. * src/qemu/qemu_driver.c (qemuFindQemuImgBinary) (qemuDomainSnapshotWriteMetadata) (qemuDomainSnapshotDiscard) (qemuDomainSnapshotForEachQcow2): Float up. (qemuDomainSnapshotDiscardDescendant): Likewise, and rename... (qemuDomainSnapshotDiscardAll): ...for generic use. (qemuDomainSnapshotDelete): Update caller. 2011-09-03 Eric Blake snapshot: teach virsh about new undefine flags Similar to 'undefine --managed-save' (commit 83e849c1), we must assume that the old API is unsafe; however, we cannot emulate metadata-only deletion on older servers. Additionally, we have the wrinkle that while virDomainUndefineFlags and managed save cleanup were introduced in 0.9.4, it wasn't until 0.9.5 that snapshots block undefine of a domain. Do the best we can given the server we are talking to. * tools/virsh.c (cmdUndefine): Add --snapshots-metadata flag. * tools/virsh.pod (undefine, destroy, shutdown): Document effect of snapshots. 2011-09-03 Eric Blake snapshot: prevent stranding snapshot data on domain destruction Just as leaving managed save metadata behind can cause problems when creating a new domain that happens to collide with the name of the just-deleted domain, the same is true of leaving any snapshot metadata behind. For safety sake, extend the semantic change of commit b26a9fa9 to also cover snapshot metadata as a reason to reject undefining an inactive domain. A future patch will make sure that shutdown of a transient domain automatically deletes snapshot metadata (whether by destroy, shutdown, or guest-initiated action). Management apps of transient domains should take care to capture xml of snapshots, if it is necessary to recreate the snapshot metadata on a later transient domain with the same name and uuid. This also documents a new flag that hypervisors can choose to support as a shortcut for taking care of the metadata as part of the undefine process; however, nontrivial driver support for these flags will be deferred to future patches. Note that ESX and VBox can never be transient; therefore, they do not have to worry about automatic cleanup after shutdown (the persistent domain still remains); likewise they never store snapshot metadata, so the undefine flag is trivial. The nontrivial work remaining is thus in the qemu driver. * include/libvirt/libvirt.h.in (VIR_DOMAIN_UNDEFINE_SNAPSHOTS_METADATA): New flag. * src/libvirt.c (virDomainUndefine, virDomainUndefineFlags): Document new limitations and flag. * src/esx/esx_driver.c (esxDomainUndefineFlags): Trivial implementation. * src/vbox/vbox_tmpl.c (vboxDomainUndefineFlags): Likewise. * src/qemu/qemu_driver.c (qemuDomainUndefineFlags): Enforce the limitations. 2011-09-03 Eric Blake snapshot: add qemu snapshot redefine support Redefining a qemu snapshot requires a bit of a tweak to the common snapshot parsing code, but the end result is quite nice. Be careful that redefinitions do not introduce circular parent chains. Also, we don't want to allow conversion between online and offline existing snapshots. We could probably do some more validation for snapshots that don't already exist to make sure they are even feasible, by parsing qemu-img output, but that can come later. * src/conf/domain_conf.h (virDomainSnapshotParseFlags): New internal flags. * src/conf/domain_conf.c (virDomainSnapshotDefParseString): Alter signature to take internal flags. * src/esx/esx_driver.c (esxDomainSnapshotCreateXML): Update caller. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotCreateXML): Likewise. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Support new public flags. 2011-09-03 Eric Blake snapshot: add qemu snapshot creation without metadata Supporting NO_METADATA on snapshot creation is interesting - we must still return a valid opaque snapshot object, but the user can't get anything out of it (unless we add a virDomainSnapshotGetName()), since it is no longer registered with the domain. Also, virsh now tries to query for secure xml, in anticipation of when we store xml inside ; for now, we can trivially support it, since we have nothing secure. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Support new flag. (qemuDomainSnapshotGetXMLDesc): Trivially support VIR_DOMAIN_XML_SECURE. 2011-09-03 Eric Blake snapshot: improve virsh snapshot-create, add snapshot-edit Wire up the new snapshot creation flags in virsh. For convenience, teach 'snapshot-current' how to make an existing snapshot become current (can be used after upgrading to newer libvirt to recover from the fact that the older libvirt lost track of the current snapshot after a restart). The snapshot-create-as command is intentionally not taught --redefine or --current, as this would imply adding a lot of other options for everything else that can appear in the xml, but which is normally read-only. Besides, redefining will usually be done on files created by snapshot-dumpxml, rather than something built up by hand on the command line. And now that we can redefine, we can edit. * tools/virsh.c (cmdSnapshotCreate): Add --redefine, --current, and --no-metadata. (cmdSnapshotCreateAs): Add --no-metadata. (cmdSnapshotCurrent): Add snapshotname to alter current snapshot. (cmdSnapshotEdit): New command. * tools/virsh.pod (snapshot-create, snapshot-create-as) (snapshot-current, snapshot-edit): Document these. 2011-09-03 Eric Blake snapshot: refactor virsh snapshot creation The next patch will make snapshot creation more complex, so it's better to avoid repetition of the complexity. * tools/virsh.c (vshSnapshotCreate): New helper function. (cmdSnapshotCreate, cmdSnapshotCreateAs): Use it. 2011-09-03 Eric Blake snapshot: allow recreation of metadata The first two flags are essential for being able to replicate snapshot hierarchies across multiple hosts, which will come in handy for supervised migrations. It also allows a management app to take a snapshot of a transient domain, save the metadata, stop the domain, recreate a new transient domain by the same name, redefine the snapshot, then revert to it. This is not quite as convenient as leaving the metadata behind after a domain is no longer around, but doing that has a few problems: 1. the libvirt API can only delete snapshot metadata if there is a valid domain handle to use to get to that snapshot object - if stale data is left behind without a domain, there is no way to request that the data be cleaned up. 2. creating a new domain with the same name but different uuid than the older domain where a snapshot existed cannot use the older snapshot data; this risks confusing libvirt, and forbidding the stale data is similar to the recent patch to forbid stale managed save. The first two flags might be useful on hypervisors with no metadata, but only for modifying the notion of the current snapshot; however, I don't know how to do that for ESX or VBox. The third flag is a convenience option, to combine a creation with a delete metadata into one step. It is trivial for hypervisors with no metadata. The qemu changes will be involved enough to warrant a separate patch. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_CREATE_REDEFINE) (VIR_DOMAIN_SNAPSHOT_CREATE_CURRENT) (VIR_DOMAIN_SNAPSHOT_CREATE_NO_METADATA): New flags. * src/libvirt.c (virDomainSnapshotCreateXML): Document them, and enforce mutual exclusion. * src/esx/esx_driver.c (esxDomainSnapshotCreateXML): Trivial implementation. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotCreateXML): Likewise. * docs/formatsnapshot.html.in: Document re-creation. 2011-09-03 Eric Blake snapshot: identify qemu snapshot roots Filtering for roots is pretty easy to do. * src/conf/domain_conf.h (virDomainSnapshotObjListGetNames) (virDomainSnapshotObjListNum): Update prototype. * src/conf/domain_conf.c (virDomainSnapshotObjListCopyNames) (virDomainSnapshotObjListGetNames, virDomainSnapshotObjListCount) (virDomainSnapshotObjListNum): Support filtering. * src/qemu/qemu_driver.c (qemuDomainSnapshotNum) (qemuDomainSnapshotListNames): Update callers. 2011-09-03 Eric Blake snapshot: reflect new dumpxml and list options in virsh New flag bits are worth exposing via virsh. In the case of snapshot-list --roots, it's possible to emulate this even when talking to an older server that lacks the bit; whereas --metadata requires a newer server. Although we don't use --security-info yet, the flag is already documented for other dumpxml operations, and turning it on now will make it useful when a future patch actually has to honor it. * tools/virsh.c (cmdSnapshotDumpXML, cmdSnapshotCurrent): Add --security-info. (cmdSnapshotList): Add --roots, --metadata. * tools/virsh.pod (snapshot-dumpxml, snapshot-current) (snapshot-list): Document these. 2011-09-03 Eric Blake snapshot: identify which snapshots have metadata To make it easier to know when undefine will fail because of existing snapshot metadata, we need to know how many snapshots have metadata. Also, it is handy to filter the list of snapshots to just those that have no parents; document that flag now, but implement it in later patches. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_LIST_ROOTS) (VIR_DOMAIN_SNAPSHOT_LIST_METADATA): New flags. * src/libvirt.c (virDomainSnapshotNum) (virDomainSnapshotListNames): Document them. * src/esx/esx_driver.c (esxDomainSnapshotNum) (esxDomainSnapshotListNames): Implement trivial flag. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotNum) (vboxDomainSnapshotListNames): Likewise. * src/qemu/qemu_driver.c (qemuDomainSnapshotNum) (qemuDomainSnapshotListNames): Likewise. 2011-09-03 Eric Blake snapshot: let qemu discard only snapshot metadata Adding this was trivial compared to the previous patch for fixing qemu snapshot deletion in the first place. * src/qemu/qemu_driver.c (qemuDomainSnapshotDiscard): Add parameter. (qemuDomainSnapshotDiscardDescendant, qemuDomainSnapshotDelete): Update callers. 2011-09-03 Eric Blake snapshot: allow deletion of just snapshot metadata A future patch will make it impossible to remove a domain if it would leave behind any libvirt-tracked metadata about snapshots, since stale metadata interferes with a new domain by the same name. But requiring snaphot contents to be deleted before removing a domain is harsh; with qemu, qemu-img can still make use of the contents after the libvirt domain is gone. Therefore, we need an option to get rid of libvirt tracking information, but not the actual contents. For hypervisors that do not track any metadata in libvirt, the implementation is trivial; all remaining hypervisors (really, just qemu) will be dealt with separately. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_DELETE_METADATA_ONLY): New flag. * src/libvirt.c (virDomainSnapshotDelete): Document it. * src/esx/esx_driver.c (esxDomainSnapshotDelete): Trivially supported when there is no libvirt metadata. * src/vbox/vbox_tmpl.c (vboxDomainSnapshotDelete): Likewise. 2011-09-03 Eric Blake snapshot: simplify acting on just children Similar to the last patch in isolating the filtering from the client actions, so that clients don't have to reinvent the filtering. * src/conf/domain_conf.h (virDomainSnapshotForEachChild): New prototype. * src/libvirt_private.syms (domain_conf.h): Export it. * src/conf/domain_conf.c (virDomainSnapshotActOnChild) (virDomainSnapshotForEachChild): New functions. (virDomainSnapshotCountChildren): Delete. (virDomainSnapshotHasChildren): Simplify. * src/qemu/qemu_driver.c (qemuDomainSnapshotReparentChildren) (qemuDomainSnapshotDelete): Likewise. 2011-09-03 Eric Blake snapshot: track current domain across deletion of children Deleting a snapshot and all its descendants had problems with tracking the current snapshot. The deletion does not necessarily proceed in depth-first order, so a parent could be deleted before a child, wreaking havoc on passing the notion of the current snapshot to the parent. Furthermore, even if traversal were depth-first, doing multiple file writes to pass current up the chain one snapshot at a time is wasteful, comparing to a single update to the current snapshot at the end of the algorithm. * src/qemu/qemu_driver.c (snap_remove): Add field. (qemuDomainSnapshotDiscard): Add parameter. (qemuDomainSnapshotDiscardDescendant): Adjust accordingly. (qemuDomainSnapshotDelete): Properly reset current. 2011-09-03 Eric Blake snapshot: avoid crash when deleting qemu snapshots This one's nasty. Ever since we fixed virHashForEach to prevent nested hash iterations for safety reasons (commit fba550f6), virDomainSnapshotDelete with VIR_DOMAIN_SNAPSHOT_DELETE_CHILDREN has been broken for qemu: it deletes children, while leaving grandchildren intact but pointing to a no-longer-present parent. But even before then, the code would often appear to succeed to clean up grandchildren, but risked memory corruption if you have a large and deep hierarchy of snapshots. For acting on just children, a single virHashForEach is sufficient. But for acting on an entire subtree, it requires iteration; and since we declared recursion as invalid, we have to switch to a while loop. Doing this correctly requires quite a bit of overhaul, so I added a new helper function to isolate the algorithm from the actions, so that callers do not have to reinvent the iteration. Note that this _still_ does not handle CHILDREN correctly if one of the children is the current snapshot; that will be next. * src/conf/domain_conf.h (_virDomainSnapshotDef): Add mark. (virDomainSnapshotForEachDescendant): New prototype. * src/libvirt_private.syms (domain_conf.h): Export it. * src/conf/domain_conf.c (virDomainSnapshotMarkDescendant) (virDomainSnapshotActOnDescendant) (virDomainSnapshotForEachDescendant): New functions. * src/qemu/qemu_driver.c (qemuDomainSnapshotDiscardChildren): Replace... (qemuDomainSnapshotDiscardDescenent): ...with callback that doesn't nest hash traversal. (qemuDomainSnapshotDelete): Use new function. 2011-09-03 Eric Blake snapshot: speed up snapshot location Each snapshot lookup was iterating over the entire hash table, O(n), instead of honing in directly on the hash key, amortized O(1). Besides, fixing this means that virDomainSnapshotFindByName can now be used inside another virHashForeach iteration (without this patch, attempts to lookup a snapshot by name during a hash iteration will fail due to nested iteration). * src/conf/domain_conf.c (virDomainSnapshotFindByName): Simplify. (virDomainSnapshotObjListSearchName): Delete unused function. 2011-09-02 Eric Blake snapshot: add snapshot-list --parent to virsh Even though I recently added 'virsh snapshot-parent', doing it one snapshot at a time is painful, so make it possible to expand the snapshot-list table at once. * tools/virsh.c (cmdSnapshotList): Add --parent. * tools/virsh.pod (snapshot-list): Document it. 2011-09-02 Eric Blake snapshot: fine-tune qemu snapshot revert states For a system checkpoint of a running or paused domain, it's fairly easy to honor new flags for altering which state to use after the revert. For an inactive snapshot, the revert has to be done while there is no qemu process, so do back-to-back transitions; this also lets us revert to inactive snapshots even for transient domains. * src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Support new flags. 2011-09-02 Eric Blake snapshot: properly revert qemu to offline snapshots Commit 5e47785 broke reverts to offline system checkpoint snapshots with older qemu, since there is no longer any code path to use qemu -loadvm on next boot. Meanwhile, reverts to offline system checkpoints have been broken for newer qemu, both before and after that commit, since -loadvm no longer works to revert to disk state without accompanying vm state. Fix both of these by using qemu-img to revert disk state. Meanwhile, consolidate the (now 3) clients of a qemu-img iteration over all disks of a VM into one function, so that any future algorithmic fixes to the FIXMEs in that function after partial loop iterations are dealt with at once. That does mean that this patch doesn't handle partial reverts very well, but we're not making the situation any worse in this patch. * src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Use qemu-img rather than 'qemu -loadvm' to revert to offline snapshot. (qemuDomainSnapshotRevertInactive): New helper. (qemuDomainSnapshotCreateInactive): Factor guts... (qemuDomainSnapshotForEachQcow2): ...into new helper. (qemuDomainSnapshotDiscard): Use it. 2011-09-02 Eric Blake snapshot: improve reverting to qemu paused snapshots If you take a checkpoint snapshot of a running domain, then pause qemu, then restore the snapshot, the result should be a running domain, but the code was leaving things paused. Furthermore, if you take a checkpoint of a paused domain, then run, then restore, there was a brief but non-deterministic window of time where the domain was running rather than paused. Fix both of these discrepancies by always pausing before restoring. Also, check that the VM is active every time lock is dropped between two monitor calls. Finally, straighten out the events that get emitted on each transition. * src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Always pause before reversion, and improve events. 2011-09-02 Eric Blake snapshot: fine-tune qemu saved images starting paused Implement the new running/paused overrides for saved state management. Unfortunately, for virDomainSaveImageDefineXML, the saved state updates are write-only - I don't know of any way to expose a way to query the current run/pause setting of an existing save image file to the user without adding a new API or modifying the domain xml of virDomainSaveImageGetXMLDesc to include a new element to reflect the state bit encoded into the save image. However, I don't think this is a show-stopper, since the API is designed to leave the state bit alone unless an explicit flag is used to change it. * src/qemu/qemu_driver.c (qemuDomainSaveInternal) (qemuDomainSaveImageOpen): Adjust signature. (qemuDomainSaveFlags, qemuDomainManagedSave) (qemuDomainRestoreFlags, qemuDomainSaveImageGetXMLDesc) (qemuDomainSaveImageDefineXML, qemuDomainObjRestore): Adjust callers. 2011-09-02 Eric Blake snapshot: expose --running and --paused in virsh Pretty straight-forward exposure of new flags. For most commands, we let the API reject mutually exclusive flags; but for save-image-edit, we do the sanity check ourselves to avoid looping on flag failure if the edit cycle is ever enhanced to allow the user to retry an edit to fix up an xml validation error. * tools/virsh.c (cmdManagedSave, cmdRestore, cmdSave) (cmdSaveImageDefine, cmdSaveImageEdit): Add new flags. * tools/virsh.pod (managedsave, restore, save, save-image-define) (save-image-edit): Document them. 2011-09-02 Eric Blake snapshot: fine-tune ability to start paused While it is nice that snapshots and saved images remember whether the domain was running or paused, sometimes the restoration phase wants to guarantee a particular state (paused to allow hot-plugging, or running without needing to call resume). This introduces new flags to allow the control, and a later patch will implement the flags for qemu. * include/libvirt/libvirt.h.in (VIR_DOMAIN_SAVE_RUNNING) (VIR_DOMAIN_SAVE_PAUSED, VIR_DOMAIN_SNAPSHOT_REVERT_RUNNING) (VIR_DOMAIN_SNAPSHOT_REVERT_PAUSED): New flags. * src/libvirt.c (virDomainSaveFlags, virDomainRestoreFlags) (virDomainManagedSave, virDomainSaveImageDefineXML) (virDomainRevertToSnapshot): Document their use, and enforce mutual exclusion. 2011-09-02 Eric Blake snapshot: better events when starting paused There are two classes of management apps that track events - one that only cares about on/off (and only needs to track EVENT_STARTED and EVENT_STOPPED), and one that cares about paused/running (also tracks EVENT_SUSPENDED/EVENT_RESUMED). To keep both classes happy, any transition that can go from inactive to paused must emit two back-to-back events - one for started and one for suspended (since later resuming of the domain will only send RESUMED, but the first class isn't tracking that). This also fixes a bug where virDomainCreateWithFlags with the VIR_DOMAIN_START_PAUSED flag failed to start paused when restoring from a managed save image. * include/libvirt/libvirt.h.in (VIR_DOMAIN_EVENT_SUSPENDED_RESTORED) (VIR_DOMAIN_EVENT_SUSPENDED_FROM_SNAPSHOT) (VIR_DOMAIN_EVENT_RESUMED_FROM_SNAPSHOT): New sub-events. * src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Use them. (qemuDomainSaveImageStartVM): Likewise, and add parameter. (qemudDomainCreate, qemuDomainObjStart): Send suspended event when starting paused. (qemuDomainObjRestore): Add parameter. (qemuDomainObjStart, qemuDomainRestoreFlags): Update callers. * examples/domain-events/events-c/event-test.c (eventDetailToString): Map new detail strings. 2011-09-02 Marc-André Lureau Learn to use spicevmc as a redirection type for usb-redir Add "redirdev" redirection device - create a new "redirdev" element for this purpose 2011-09-02 Marc-André Lureau qemu: Don't append 0 at usb id, so that it is compatible with legacy -usb QEMU uses USB bus name "usb.0" when using the legacy -usb argument. If we want to allow USB devices to specify their addresses with legacy -usb, we should either in case of legacy bus name drop the 0 from the address bus, or just drop the 0 from device id. This patch does the later. Another solution would be to permit addressing on non-legacy USB controllers only. 2011-09-02 Marc-André Lureau qemu: don't reserve slot 1 if a PIIX3 USB controller is defined there Applies only to piix3 and check if piix3 controller is on correct address, or report error 2011-09-02 Marc-André Lureau Modify USB port to be defined as a port path So that devices can be attached to hubs. Example, to attach to first port of a usb-hub on port 1.
also add a test entry 2011-09-02 Marc-André Lureau Add USB hub device domain parsing and serialization code, qemu driver backend and a couple of test 2011-09-02 Marc-André Lureau Add USB companion controllers support Companion controllers take an extra 'master' attribute to associate them. Also add tests for this 2011-09-02 Marc-André Lureau USB devices gain a new USB address child element Expand the domain and the QEmu driver code Adds a couple of tests test: USB controller can have a PCI address child element add a few tests for the new constructs 2011-09-02 Marc-André Lureau Add a new controller type 'usb' with optionnal 'model' The model by default is piix3-uchi. Example: 2011-09-02 Marc-André Lureau Add USB controller models List is: piix3-uhci piix4-uhci ehci ich9-ehci1 ich9-uhci1 ich9-uhci2 ich9-uhci3 vt82c686b-uhci pci-ohci Rename virDomainControllerModel to virDomainControllerModelSCSI Since we are about to add USB controller support let's remove the ambiguity Add various USB devices QEMU_CAPS 2011-09-02 Eric Blake snapshot: fix corner case on OOM during creation Commit 6766ff10 introduced a corner case bug with snapshot creation: if a snapshot is created, but then we hit OOM while trying to create the return value of the function, then we have polluted the internal directory with the snapshot metadata with no way to clean it up from the running libvirtd. * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Don't write metadata file on OOM condition. 2011-09-02 Eric Blake tests: add missing files Created by copying from qemuxml2argv-disk-drive-v2-wb.*, then s/writeback/directsync/. Hopefully this matches Osier's intentions. * tests/qemuxml2argvdata/qemuxml2argv-disk-drive-cache-directsync.args: * tests/qemuxml2argvdata/qemuxml2argv-disk-drive-cache-directsync.xml: Add missing files needed by 'make check'. 2011-09-02 Osier Yang Add directsync cache mode support for disk driver Newer QEMU introduced cache=directsync for -drive, this patchset is to expose it in libvirt layer. * Introduced a new QEMU capability flag ($prefix_CACHE_DIRECTSYNC), As even $prefix_CACHE_V2 is set, we can't known if directsync is supported. 2011-09-02 Osier Yang storage: Add virsh support for fs pool formating 2011-09-02 Osier Yang storage: Add fs pool formatting This patch adds the ability to make the filesystem for a filesystem pool during a pool build. The patch adds two new flags, no overwrite and overwrite, to control when mkfs gets executed. By default, the patch preserves the current behavior, i.e., if no flags are specified, pool build on a filesystem pool only makes the directory on which the filesystem will be mounted. If the no overwrite flag is specified, the target device is checked to determine if a filesystem of the type specified in the pool is present. If a filesystem of that type is already present, mkfs is not executed and the build call returns an error. Otherwise, mkfs is executed and any data present on the device is overwritten. If the overwrite flag is specified, mkfs is always executed, and any existing data on the target device is overwritten unconditionally. 2011-09-02 Osier Yang storage: Add mkfs and libblkid to build system API: Init conn in case of it might be used uninitialized There is a goto before "conn" is initialized. 2011-09-02 Eric Blake qemu: detect incomplete save files Several users have reported problems with 'virsh start' failing because it was encountering a managed save situation where the managed save file was incomplete. Be more robust to this by using two different magic numbers, so that newer libvirt can gracefully handle an incomplete file differently than a complete one, while older libvirt will at least fail up front rather than trying to load only to have qemu fail at the end. Managed save is a convenience - it exists to preserve as much state as possible; if the state was not preserved, it is reasonable to just log that fact, then proceed with a fresh boot. On the other hand, user saves are under user control, so we must fail, but by making the failure message distinct, the user can better decide how to handle the situation of an incomplete save file. * src/qemu/qemu_driver.c (QEMUD_SAVE_PARTIAL): New define. (qemuDomainSaveInternal): Use it to mark incomplete images. (qemuDomainSaveImageOpen, qemuDomainObjRestore): Add parameter that controls what to do with partial images. (qemuDomainRestoreFlags, qemuDomainSaveImageGetXMLDesc) (qemuDomainSaveImageDefineXML, qemuDomainObjStart): Update callers. Based on an initial idea by Osier Yang. 2011-09-02 Eric Blake qemu: refactor file opening In a SELinux or root-squashing NFS environment, libvirt has to go through some hoops to create a new file that qemu can then open() by name. Snapshots are a case where we want to guarantee an empty file that qemu can open; also, reopening a save file to convert it from being marked partial to complete requires a reopen to avoid O_DIRECT headaches. Refactor some existing code to make it easier to reuse in later patches. * src/qemu/qemu_migration.h (qemuMigrationToFile): Drop parameter. * src/qemu/qemu_migration.c (qemuMigrationToFile): Let cgroup do the stat, rather than asking caller to do it and pass info down. * src/qemu/qemu_driver.c (qemuOpenFile): New function, pulled from... (qemuDomainSaveInternal): ...here. (doCoreDump, qemuDomainSaveImageOpen): Use it here as well. 2011-09-02 Wen Congyang reserve slot 1 on pci bus0 After supporting multi function pci device, we only reserve function 1 on slot 1. The user can use the other function on slot 1 in the xml config file. We should detect this wrong usage. 2011-09-02 Xu He Jie libvirtd: create run dir when running at non-root user When libvirtd is running at non-root user, it won't create ${HOME}/.libvirt. It will show error message: 17:44:16.838: 7035: error : virPidFileAcquirePath:322 : Failed to open pid file 2011-09-02 Scott Moser lxc: do not require 'ifconfig' or 'ipconfig' in container Currently, the lxc implementation invokes 'ip' and 'ifconfig' commands inside a container using 'virRun'. That has the side effect of requiring those commands to be present and to function in a manner consistent with the usage. Some small roots (such as ttylinux) may not have 'ip' or 'ifconfig'. This patch replaces the use of these commands with usage of netdevice. The result is that lxc containers do not have to implement those commands, and lxc in libvirt is only dependent on the netdevice interface. I've tested this patch locally against the ubuntu libvirt version enough to verify its generally sane. I attempted to build upstream today, but failed with: /usr/bin/ld: ../src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_domain.o): undefined reference to symbol 'xmlXPathRegisterNs@@LIBXML2_2.4.30 Thats probably a local issue only, but I wanted to get this patch up and see what others thought of it. This is ubuntu bug https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/828211 . 2011-09-02 Serge Hallyn Create ptmx as a device Hi, I'm seeing an issue with udev and libvirt-lxc. Libvirt-lxc creates /dev/ptmx as a symlink to /dev/pts/ptmx. When udev starts up, it checks the device type, sees ptmx is 'not right', and replaces it with a 'proper' ptmx. In lxc, /dev/ptmx is bind-mounted from /dev/pts/ptmx instead of being symlinked, so udev sees the right device type and leaves it alone. A patch like the following seems to work for me. Would there be any objections to this? >From 4c5035de52de7e06a0de9c5d0bab8c87a806cba7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Wed, 31 Aug 2011 18:15:54 +0000 Subject: [PATCH 1/1] make ptmx a bind mount rather than symlink udev on some systems checks the device type of /dev/ptmx, and replaces it if not as expected. The symlink created by libvirt-lxc therefore gets replaced. By creating it as a bind mount, the device type is correct and udev leaves it alone. 2011-09-02 Adam Litke BlockPull: Set initial bandwidth limit if specified The libvirt BlockPull API supports the use of an initial bandwidth limit but the qemu block_stream API does not. To get the desired behavior we use the two APIs strung together: first BlockPull, then BlockJobSetSpeed. We can do this at the driver level to avoid duplicated code in each monitor path. 2011-09-02 Adam Litke BlockJob: Bandwidth parameter is in MB when using text monitor Due to an unfortunate precedent in qemu, the units for the bandwidth parameter to block_job_set_speed are different between the text monitor and the qmp monitor. While the qmp monitor uses bytes/s, the text monitor expects MB/s. Correct the units for the text interface. 2011-09-02 Jim Fehlig Increase size of buffer for xend response On systems with many pcpus, the sexpr returned by xend can be quite large for dom0 when it is configured to have #vcpus = #pcpus (default). E.g. on a 80 pcpu system, where dom0 had 80 vcpus, the sexpr details for dom0 was 73817 bytes! Increase maximum buffer size to 256k. 2011-09-02 Jim Fehlig Don't overwrite errors from xend_{get,req} xenDaemonDomainFetch() was overwriting errors reported by xend_get() and xend_req(). E.g. without patch error: failed Xen syscall xenDaemonDomainFetch failed to find this domain with patch error: internal error Xend returned HTTP Content-Length of 73817, which exceeds maximum of 65536 2011-09-02 Jim Fehlig Change my email domain I'd like to change my primary email address to jfehlig@suse.com. This patch changes it in AUTHORS and .mailmap. 2011-09-01 Eric Blake virsh: improve send-key documentation The 'virsh man' description of send-key was incomplete and used the old style (literal 'optional name' instead of '[name]' metasyntax). Meanwhile, none of the other virsh help texts include examples, so I moved it out of virsh help and into the man page. * tools/virsh.pod (send-key): Give better details. * tools/virsh.c (info_send_key): Drop example from here. 2011-09-01 Eric Blake virsh: support 'virsh start --force-boot' on older servers Managed save was added in 0.8.0, virDomainCreateWithFlags in 0.8.2, and FORCE_BOOT in 0.9.5. The virsh flag is more useful if we emulate it for all older servers (note that if a hypervisor fails the query for a managed save image, then it does not have one to be removed, so the flag can be safely ignored). * tools/virsh.c (cmdStart): Add emulation for new flag. 2011-09-01 Alex Jia virsh: avoid memory leak on cmdVolCreateAs * tools/virsh.c: fix memory leak on cmdVolCreateAs function. * Detected in valgrind run: ==4746== ==4746== 48 (40 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 26 of 52 ==4746== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==4746== by 0x4C76E51: virAlloc (memory.c:101) ==4746== by 0x4CD9418: virGetStoragePool (datatypes.c:592) ==4746== by 0x4D21367: remoteStoragePoolLookupByName (remote_driver.c:4126) ==4746== by 0x4CE42B0: virStoragePoolLookupByName (libvirt.c:10232) ==4746== by 0x40C276: vshCommandOptPoolBy (virsh.c:13660) ==4746== by 0x40CA37: cmdVolCreateAs (virsh.c:8094) ==4746== by 0x412AF2: vshCommandRun (virsh.c:13770) ==4746== by 0x422F11: main (virsh.c:15127) ==4746== ==4746== 1,011 bytes in 1 blocks are definitely lost in loss record 45 of 52 ==4746== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==4746== by 0x4A06167: realloc (vg_replace_malloc.c:525) ==4746== by 0x4C76ECB: virReallocN (memory.c:161) ==4746== by 0x4C60319: virBufferGrow (buf.c:72) ==4746== by 0x4C606AA: virBufferAdd (buf.c:106) ==4746== by 0x40CB37: cmdVolCreateAs (virsh.c:8118) ==4746== by 0x412AF2: vshCommandRun (virsh.c:13770) ==4746== by 0x422F11: main (virsh.c:15127) ==4746== ==4746== LEAK SUMMARY: ==4746== definitely lost: 1,051 bytes in 2 blocks ==4746== indirectly lost: 8 bytes in 1 blocks ==4746== possibly lost: 0 bytes in 0 blocks ==4746== still reachable: 390,767 bytes in 1,373 blocks ==4746== suppressed: 0 bytes in 0 blocks * How to reproduce? % valgrind -v --leak-check=full virsh vol-create-as default foo.img 10M \ --allocation 0 --format qcow2 --backing-vol bar.img Notes: bar.img doesn't exist. 2011-09-01 Eric Blake build: fix 'make check' with pdwtags Problem introduced by commit b12354b. * src/remote_protocol-structs: Remove spurious blank line. 2011-09-01 Jim Fehlig virsh: Expose virDomainMigrateGetMaxSpeed API 2011-09-01 Jim Fehlig Add public API for getting migration speed Includes impl of python binding since the generator was not able to cope. Note: Requires gendispatch.pl patch from Matthias Bolte https://www.redhat.com/archives/libvir-list/2011-August/msg01367.html 2011-09-01 Eric Blake virsh: prefer unsigned flags virsh had some leftover 'int flags', and even an 'int flag' declaration, compared to our preferred style of 'unsigned int flags'. * tools/virsh.c (cmdUndefine, cmdSave, cmdSaveImageDumpxml) (cmdSaveImageEdit, cmdManagedSave, cmdRestore, cmdDump) (cmdVcpuPin, cmdSetvcpus, cmdSetmem, cmdSetmaxmem, cmdDumpXML) (cmdDomXMLFromNative, cmdDomXMLToNative, doMigrate) (cmdInterfaceEdit, cmdInterfaceDumpXML, cmdEdit): Match coding style for flags. (struct vshComdOptDef): Rename field member. (vshCmddefOptParse, vshCmddefHelp): Adjust clients. 2011-09-01 Daniel P. Berrange Fix tracking of RPC messages wrt streams Commit 2c85644b0b51fbe5b6244e6773531af29933a727 attempted to fix a problem with tracking RPC messages from streams by doing - if (msg->header.type == VIR_NET_REPLY) { + if (msg->header.type == VIR_NET_REPLY || + (msg->header.type == VIR_NET_STREAM && + msg->header.status != VIR_NET_CONTINUE)) { client->nrequests--; In other words any stream packet, with status NET_OK or NET_ERROR would cause nrequests to be decremented. This is great if the packet from from a synchronous virStreamFinish or virStreamAbort API call, but wildly wrong if from a server initiated abort. The latter resulted in 'nrequests' being decremented below zero. This then causes all I/O for that client to be stopped. Instead of trying to infer whether we need to decrement the nrequests field, from the message type/status, introduce an explicit 'bool tracked' field to mark whether the virNetMessagePtr object is subject to tracking. Also add a virNetMessageClear function to allow a message contents to be cleared out, without adversely impacting the 'tracked' field as a naive memset() would do * src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Add a 'bool tracked' field and virNetMessageClear() API * daemon/remote.c, daemon/stream.c, src/rpc/virnetclientprogram.c, src/rpc/virnetclientstream.c, src/rpc/virnetserverclient.c, src/rpc/virnetserverprogram.c: Switch over to use virNetMessageClear() and pass in the 'bool tracked' value when creating messages. 2011-09-01 Daniel P. Berrange Avoid use-after-free on streams, due to message callbacks When sending outbound stream RPC messages, a callback is used to re-enable stream data transmission. If the stream aborts while one of these messages is outstanding, the stream may have been free'd by the time it is invoked. This results in a use-after-free error * daemon/stream.c: Ref-count streams to avoid use-after-free 2011-09-01 Daniel P. Berrange Fix parted sector size assumption Parted does not report disk size in 512 byte units, but rather the disks' logical sector size, which with modern drives might be 4k. * src/storage/parthelper.c: Remove hardcoded 512 byte sector size 2011-09-01 Osier Yang qemu: Fix a regression of domain save * src/qemu/qemu_driver.c - qemuDomainSaveInternal: Return directly will keep the domain object locked, introduced by 173015bec6a. Revert "test: Cleanup improper VIR_ERR_NO_SUPPORT use" This reverts commit 172214bd304ff958160307be2efd6614e9868946. storage: Fix incorrect error codes Commit 0376f4a69b intended to fix incorrect use of VIR_ERR_NO_SUPPORT, but replacing it with VIR_ERR_OPERATION_INVALID is not proper either. remote: Fix incorrect error codes Introduced by d4b53ef6c. For "no internalFlags support", the error code is changed into INTERNAL_ERROR. nodeinfo: Fix incorrect error codes Introduced by 5e495c8b, except the ones for checking if numa is supported by host, all the NO_SUPPORT are changed back. For the ones about numa checking, change them into INTERNAL_ERROR. lxc: Fix incorrect changes on error codes. Fix incorrect changes introduced by commit 6ac47762bb9. conf: Substitute OPERATION_INVALID with INTERNAL_ERROR 2011-08-31 Daniel P. Berrange Stop libxl driver polluting logs on non-Xen hosts If the libxl driver is compiled in, then everytime libvirtd starts up on a non-Xen Dom0 host, it logs a error message. Since this is an expected condition, we should not log at 'error' level, only 'info'. * src/libxl/libxl_driver.c: Lower log level for certain expected errors during driver init 2011-08-31 Daniel P. Berrange Fix memory leak parsing 'relabel' attribute in domain security XML * src/conf/domain_conf.c: Free the 'relabel' attribute 2011-08-31 Daniel P. Berrange Fix memory leak dispatching domain events When dispatching domain events we will create an XDR struct containing the event info. Some of this data may be allocated on the heap and so must be freed. The graphics event dispatcher had a broken attempt to free one field, but missed others. All the events have a dom->name string that needs freeing. The code should have used the xdr_free() procedure for doing all this * daemon/remote.c: Use xdr_free after dispatching events 2011-08-31 Daniel P. Berrange Don't leak memory if a cgroup is mounted multiple times It is possible (expected/likely in Fedora 15) for a cgroup controller to be mounted in multiple locations at the same time, due to bind mounts. Currently we leak memory if this happens, because we overwrite the previous 'mountPoint' string. Instead just accept the first match we find. * src/util/cgroup.c: Only accept first match for a cgroup controller mount 2011-08-31 Eric Blake security: fix build Regression introduced in commit 183383889. * src/libvirt_private.syms (security_manager.h): Drop deleted symbol. Detected by build-bot. 2011-08-31 Guannan Ren stream: remove redundant reference to client while sending stream data *daemon/stream.c: remove virNetServerClientRef() 2011-08-31 Daniel P. Berrange Remove bogus virSecurityManagerSetProcessFDLabel method The virSecurityManagerSetProcessFDLabel method was introduced after a mis-understanding from a conversation about SELinux socket labelling. The virSecurityManagerSetSocketLabel method should have been used for all such scenarios. * src/security/security_apparmor.c, src/security/security_apparmor.c, src/security/security_driver.h, src/security/security_manager.c, src/security/security_manager.h, src/security/security_selinux.c, src/security/security_stack.c: Remove SetProcessFDLabel driver 2011-08-31 Daniel P. Berrange Fix sanlock socket security labelling It is not possible to change the label of a TCP socket once it has been opened. When creating a TCP socket care must be taken to ensure the socket creation label is set & then cleared. Remove the bogus call to virSecurityManagerSetProcessFDLabel from the lock driver guest setup code and instead make use of virSecurityManagerSetSocketLabel 2011-08-31 Daniel P. Berrange Fix incorrect path length check in sanlock lockspace setup The code for creating a sanlock lockspace accidentally used SANLK_NAME_LEN instead of SANLK_PATH_LEN for a size check. This meant disk paths were limited to 48 bytes ! * src/locking/lock_driver_sanlock.c: Fix disk path length check 2011-08-30 Eric Blake snapshot: forbid snapshot on autodestroy domain There is no reason to forbid pausing an autodestroy domain (not to mention that 'virsh start --paused --autodestroy' succeeds in creating a paused autodestroy domain). Meanwhile, qemu was failing to enforce the API documentation that autodestroy domains cannot be saved. And while the original documentation only mentioned save/restore, snapshots are another form of saving that are close enough in semantics as to make no sense on one-shot domains. * src/qemu/qemu_driver.c (qemudDomainSuspend): Drop bogus check. (qemuDomainSaveInternal, qemuDomainSnapshotCreateXML): Forbid saves of autodestroy domains. * src/libvirt.c (virDomainCreateWithFlags, virDomainCreateXML): Document snapshot interaction. 2011-08-30 Philipp Hahn Fix error detection in device change According to qemu-kvm/qerror.c all messages start with a capital "Device ", but the current code only scans for the lower case "device ". This results in "virDomainUpdateDeviceFlags()" to not detect locked CD-ROMs and reporting success even in the case of a failure: # virsh qemu-monitor-command "$VM" change\ drive-ide0-0-0\ \"/var/lib/libvirt/images/ucs_2.4-0-sec4-20110714145916-dvd-amd64.iso\" Device 'drive-ide0-0-0' is locked # virsh update-device "$VM" /dev/stdin <<<"
" Device updated successfully 2011-08-30 Eric Blake start: allow discarding managed save There have been several instances of people having problems with a broken managed save file, and not aware that they could use 'virsh managedsave-remove dom' to fix things. Making it possible to do this as part of starting a domain makes the same functionality easier to find, and one less API call. * include/libvirt/libvirt.h.in (VIR_DOMAIN_START_FORCE_BOOT): New flag. * src/libvirt.c (virDomainCreateWithFlags): Document it. * src/qemu/qemu_driver.c (qemuDomainObjStart): Alter signature. (qemuAutostartDomain, qemuDomainStartWithFlags): Update callers. * tools/virsh.c (cmdStart): Expose it in virsh. * tools/virsh.pod (start): Document it. 2011-08-30 Eric Blake build: simplify use of verify Back in 2008 when this line of util.h was written, gnulib's verify module didn't allow the use of multiple verify() in one file in combination with our choice of gcc -W options. But that has since been fixed in gnulib, and newer gnulib even maps verify() to the C1x feature of _Static_assert, which gives even nicer diagnostics with a new enough compiler, so we might as well go with the simpler verify(). * src/util/util.h (VIR_ENUM_IMPL): Use simpler verify, now that gnulib module is smarter. 2011-08-30 Eric Blake qemu: properly label outgoing pipe for tunneled migration Commit 3261761 made it possible to use pipes instead of sockets for outgoing tunneled migration; however, it caused a regression because the pipe was never given a SELinux label. * src/qemu/qemu_migration.c (doTunnelMigrate): Label outgoing pipe. 2011-08-29 Guannan Ren rpc: fix a typo in debugging log in virNetServerProgramSendStreamData The bufferOffset has been initialized to zero in virNetMessageEncodePayloadRaw(), so, we use bufferLength to represent the length of message which is going to be sent to client side. 2011-08-27 Eric Blake maint: fix spelling errors on lose * docs/drvqemu.html.in: Fix typo. * src/libvirt.c (virDomainCreateXML, virDomainCreateWithFlags): Likewise. 2011-08-26 Jim Fehlig Fix generator to cope with call-by-ref long types From: Matthias Bolte Tested-by: Jim Fehlig Matthias provided this patch to fix an issue I encountered in the generator with APIs containing call-by-ref long type, e.g. int virDomainMigrateGetMaxSpeed(virDomainPtr domain, unsigned long *bandwidth, unsigned int flags); 2011-08-26 Matthias Bolte hyperv: Add basic documentation 2011-08-26 Matthias Bolte hyperv: Add basic driver for Microsoft Hyper-V Domain listing, basic information retrieval and domain life cycle management is implemented. But currently the domain XML output lacks the complete devices section. The driver uses OpenWSMAN to directly communicate with a Hyper-V server over its WS-Management interface exposed via Microsoft WinRM. The driver is based on the work of Michael Sievers. This started in the same master program project group at the University of Paderborn as the ESX driver. See Michael's blog for details: http://hyperv4libvirt.wordpress.com/ 2011-08-26 Matthias Bolte hyperv: Add OpenWSMAN based client for the Hyper-V WMI API Add a generator script to generate the structs and serialization information for OpenWSMAN. openwsman.h collects workarounds for problems in OpenWSMAN <= 2.2.6. There are also disabled sections that would use ws_serializer_free_mem but can't because it's broken in OpenWSMAN <= 2.2.6. Patches to fix this have been posted upstream. 2011-08-26 Matthias Bolte hyperv: Add driver skeleton hyperv: Add configure check for OpenWSMAN 2011-08-26 Taku Izumi schedinfo: update man page about virsh schedinfo command This patch updates the man page about virsh schedinfo command. - fix typo: 1844674407370955 -> 18446744073709551 - describe the value 0 of vcpu_period and vcpu_quota parameters 2011-08-26 KAMEZAWA Hiroyuki Fix persistent migration config save When a user migrates a domain by command as libvirt saves vm's domain XML config in destination host after migration. But it saves vm->def. Then, the saved XML contains some garbage. ^^^^^^^^ ... ^^^^^^^^^^^^^^^^^ Avoid saving unnecessary things by saving persistent vm definition. 2011-08-26 Jiri Denemark rpc: Don't close connection if program is unknown In case we add a new program in the future (we did that in the past and we are going to do it again soon) current daemon will behave badly with new client that wants to use the new program. Before the RPC rewrite we used to just send an error reply to any request with unknown program. With the RPC rewrite in 0.9.3 the daemon just closes the connection through which such request was sent. This patch fixes this regression. 2011-08-26 Michal Privoznik remote: Refuse connecting to remote socket If users wants to connect to remote unix socket, e.g. 'qemu+unix:///system' currently the part is ignored, ending up connecting to localhost. Connecting to remote socket is not supported and user should have used TLS/TCP/SSH instead. virterror: Fix error message for VIR_ERR_INVALID_ARG When a detail message is presented, nobody expects prefix 'invalid argument in' but something more general, like 'invalid argument:'. 2011-08-26 Daniel P. Berrange Detect errors from the 'sendkey' command On success, the 'sendkey' command does not return any data, so any data in the reply should be considered to be an error message * src/qemu/qemu_monitor_text.c: Treat non-"" reply data as an error message for 'sendkey' command 2011-08-26 Daniel P. Berrange Fix keymap used to talk with QEMU The QEMU 'sendkey' command expects keys to be encoded in the same way as the RFB extended keycode set. Specifically it wants extended keys to have the high bit of the first byte set, while the Linux XT KBD driver codeset uses the low bit of the second byte. To deal with this we introduce a new keymap 'RFB' and use that in the QEMU driver * include/libvirt/libvirt.h.in: Add VIR_KEYCODE_SET_RFB * src/qemu/qemu_driver.c: Use RFB keycode set instead of XT KBD * src/util/virkeycode-mapgen.py: Auto-generate the RFB keycode set from the XT KBD set * src/util/virkeycode.c: Add RFB keycode entry to table. Add a verify check on cardinality of the codeOffset table 2011-08-26 Jiri Denemark virsh: Clarify documentation of -d option The default is 4, not 0. qemu: Correctly label migration TCP socket security: Introduce SetSocketLabel This API labels all sockets created until ClearSocketLabel is called in a way that a vm can access them (i.e., they are labeled with svirt_t based label in SELinux). security: Rename SetSocketLabel APIs to SetDaemonSocketLabel The APIs are designed to label a socket in a way that the libvirt daemon itself is able to access it (i.e., in SELinux the label is virtd_t based as opposed to svirt_* we use for labeling resources that need to be accessed by a vm). The new name reflects this. Ignore unused streams in virStreamAbort When virStreamAbort is called on a stream that has not been used yet, quite confusing error is returned: "this function is not supported by the connection driver". Let's just ignore such streams as there's nothing to abort anyway. Do not try to cancel non-existent migration on source If migration failed on source daemon, the migration is automatically canceled by the daemon itself. Thus we don't need to call virDomainMigrateConfirm3(cancelled=1). Calling it doesn't cause any harm but the resulting error message printed in logs may confuse people. 2011-08-25 Eric Blake snapshot: track current snapshot across restarts Audit all changes to the qemu vm->current_snapshot, and make them update the saved xml file for both the previous and the new snapshot, so that there is always at most one snapshot with 1 in the xml, and that snapshot is used as the current snapshot even across libvirtd restarts. This patch does not fix the case of virDomainSnapshotDelete(,CHILDREN) where one of the children is the current snapshot; that will be later. * src/conf/domain_conf.h (_virDomainSnapshotDef): Alter member type and name. * src/conf/domain_conf.c (virDomainSnapshotDefParseString) (virDomainSnapshotDefFormat): Update clients. * docs/schemas/domainsnapshot.rng: Tighten rng. * src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Reload current snapshot. (qemuDomainSnapshotCreateXML, qemuDomainRevertToSnapshot) (qemuDomainSnapshotDiscard): Track current snapshot. 2011-08-25 Eric Blake snapshot: only pass snapshot to qemu command line when reverting Changing the current vm, and writing that change to the file system, all before a new qemu starts, is risky; it's hard to roll back if starting the new qemu fails for some reason. Instead of abusing vm->current_snapshot and making the command line generator decide whether the current snapshot warrants using -loadvm, it is better to just directly pass a snapshot all the way through the call chain if it is to be loaded. This frees up the last use of snapshot->def->active for qemu's use, so the next patch can repurpose that field for tracking which snapshot is current. * src/qemu/qemu_command.c (qemuBuildCommandLine): Don't use active field of snapshot. * src/qemu/qemu_process.c (qemuProcessStart): Add a parameter. * src/qemu/qemu_process.h (qemuProcessStart): Update prototype. * src/qemu/qemu_migration.c (qemuMigrationPrepareAny): Update callers. * src/qemu/qemu_driver.c (qemudDomainCreate) (qemuDomainSaveImageStartVM, qemuDomainObjStart) (qemuDomainRevertToSnapshot): Likewise. (qemuDomainSnapshotSetCurrentActive) (qemuDomainSnapshotSetCurrentInactive): Delete unused functions. 2011-08-25 Eric Blake snapshot: don't leak resources on qemu snapshot failure https://bugzilla.redhat.com/show_bug.cgi?id=727709 mentions that if qemu fails to create the snapshot (such as what happens on Fedora 15 qemu, which has qmp but where savevm is only in hmp, and where libvirt is old enough to not try the hmp fallback), then 'virsh snapshot-list dom' will show a garbage snapshot entry, and the libvirt internal directory for storing snapshot metadata will have a bogus file. This fixes the fallout bug of polluting the snapshot-list with garbage on failure (the root cause of the F15 bug of not having fallback to hmp has already been fixed in newer libvirt releases). * src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Allocate memory before making snapshot, and cleanup on failure. Don't dereference NULL if transient domain exited during snapshot creation. 2011-08-25 Alex Jia libvirt: avoid dead store in virDomainMigrateVersion3 * src/qemu/qemu_migration.c: avoid dead 'ret' assignment and silence clang warning. Detected by ccc-analyzer: libvirt.c:4277:5: warning: Value stored to 'ret' is never read ret = domain->conn->driver->domainMigrateConfirm3 ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2011-08-25 Alex Jia qemu: avoid dead store in doPeer2PeerMigrate3 * src/qemu/qemu_migration.c: avoid dead 'ret' assignment and silence clang warning. Detected by ccc-analyzer: CC libvirt_driver_qemu_la-qemu_migration.lo qemu/qemu_migration.c:2046:5: warning: Value stored to 'ret' is never read ret = qemuMigrationConfirm(driver, sconn, vm, ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2011-08-25 Daniel P. Berrange Fix command test wrt gnutls initialize & fix debugging The VIR_TEST_DEBUG and VIR_TEST_VERBOSE env vars did not work because we replaced 'environ' with 'newenv'. Simply calling virTestGetDebug/Verbose() before replacing the 'environ' ensures we have processed the env variables. The gnutls initialization code opens /dev/urandom and keeps that FD around for later use. We have code which kills off FDs 3-5 to avoid interfereing with our test case. Move the virInitialize call before this point, so it kills off the gnutls /dev/urandom FD which is irrelevant for testing purposes * tests/commandtest.c: Fix test debugging & make it robust against opened FDs 2011-08-25 Michal Privoznik daemon: Move TLS initialization to virInitialize My previous patch 74c75671331d284e1f777f9692b72e9737520bf0 introduced a regression by removing TLS initialization from client. 2011-08-24 Laine Stump util: only fchown newly created files in virFileOpenAs virFileOpenAs takes desired uid:gid as arguments, and not only uses them for a fork/setuid/setgid when retrying failed open operations, but additionally always forces the opened file to be owned by the given uid:gid. One example of the problems this causes is that, when restoring a domain from a file that is owned by the qemu user, opening the file chowns it to root. if dynamic_ownership=1 this is coincidentally expected, but if dynamic_ownership=0, no existing file should ever have its ownership changed. This patch adds an extra check before calling fchown() - it only does it if O_CREAT was passed to virFileOpenAs() in the openflags. 2011-08-24 Shradha Shah qemu: fix off-by-one in pci slot reservation 2011-08-24 Steve Hodgson maint: attribute second author of previous patch 2011-08-24 Shradha Shah pci: fix pciDeviceListSteal on multiple devices pciDeviceListSteal(pcidevs, dev) removes dev from pcidevs reducing the length of pcidevs, so moving onto what was the next dev is wrong. Instead callers should pop entry 0 repeatedly until pcidevs is empty. 2011-08-24 Eric Blake libvirt: log all flags parameters I was testing a virsh patch, and wanted to see if I had passed the flags I thought. But with LIBVIRT_DEBUG in the environment, I just saw: 14:24:52.359: 15022: debug : virDomainSnapshotNum:15586 : dom=0xc9c180, (VM: name=rhel_6-64, uuid=48f8e8e7-e14f-0e14-02f0-ce71997bdcab), including a trailing space. This fixes the issues. * src/libvirt.c: Log flag parameters, even if currently unused. (VIR_DOMAIN_DEBUG_0): Drop trailing comma in log. (VIR_DOMAIN_DEBUG_1): Split guts into... (VIR_DOMAIN_DEBUG_2): ...new macro. 2011-08-24 Eric Blake virsh: add list --managed-save Knowing whether 'virsh start' will resume a saved image or do a fresh boot is useful enough to expose via 'virsh list'. Also, translate the state column. * tools/virsh.c (cmdList): add --managed-save flag * tools/virsh.pod (list): Document it. Based on a suggestion by Miklos Vajna. 2011-08-24 Matthias Bolte esx: Use $(PYTHON) instead of the shebang to run the generator 2011-08-24 Osier Yang qemu: Report error if qemu monitor command not found for BlockJob * src/qemu/qemu_monitor_json.c: Handle error "CommandNotFound" and report the error. * src/qemu/qemu_monitor_text.c: If a sub info command is not found, it prints the output of "help info", for other commands, "unknown command" is printed. Without this patch, libvirt always report: An error occurred, but the cause is unknown This patch was adapted from a patch by Osier Yang to break out detection of unrecognized text monitor commands into a separate function. 2011-08-23 Matthias Bolte esx: Refactor a repeated string in the generator 2011-08-23 Eric Blake maint: fix comment typos * src/qemu/qemu_driver.c (qemuDomainSaveInternal): Fix typo. * src/conf/domain_event.c (virDomainEventDispatchMatchCallback): Likewise. * daemon/libvirtd.c (daemonRunStateInit): Likewise. * src/lxc/lxc_container.c (lxcContainerChildMountSort): Likewise. * src/util/virterror.c (virCopyError, virRaiseErrorFull): Likewise. * src/xenxs/xen_sxpr.c (xenParseSxprSound): Likewise. 2011-08-23 Osier Yang virsh: Substitute goto label out with cleanup Introduced by 241cbc13a, pushed under urgent rule. 2011-08-23 Osier Yang virsh: Do not try to free domain if it is NULL Without these patch, there will be error like below if domain is NULL. error: invalid domain pointer in virDomainFree Which is useless. 2011-08-23 Osier Yang xen: Cleanup improper VIR_ERR_NO_SUPPORT use test: Cleanup improper VIR_ERR_NO_SUPPORT use storage: Cleanup improper VIR_ERR_NO_SUPPORT use remote: Cleanup improper VIR_ERR_NO_SUPPORT use 2011-08-23 Osier Yang qemu: Cleanup improper VIR_ERR_NO_SUPPORT use * src/qemu/qemu_command.c: s/VIR_ERR_NO_SUPPORT/VIR_ERR_CONFIG_UNSUPPORTED/ * src/qemu/qemu_driver.c: s/VIR_ERR_NO_SUPPORT/VIR_ERR_OPERATION_INVALID/ * src/qemu/qemu_process.c: s/VIR_ERR_NO_SUPPORT/VIR_ERR_OPERATION_INVALID/ 2011-08-23 Osier Yang nodeinfo: Cleanup improper VIR_ERR_NO_SUPPORT use 2011-08-23 Osier Yang lxc: Cleanup improper VIR_ERR_NO_SUPPORT use s/VIR_ERR_NO_SUPPORT/VIR_ERR_OPERATION_INVALID/ Special case is changes on lxcDomainInterfaceStats, if it's not implemented on the platform, prints error like: lxcError(VIR_ERR_OPERATION_INVALID, "%s", _("interface stats not implemented on this platform")); As the function is supported by driver actually, error like VIR_ERR_NO_SUPPORT is confused. 2011-08-23 Osier Yang conf: Cleanup improper use of VIR_ERR_NO_SUPPORT in node_device_conf qemu: Substitute VIR_ERR_NO_SUPPORT with VIR_ERR_OPERATION_INVALID * src/qemu/qemu_monitor_text.c: Error like "this function is not supported by the connection driver" is confused obviously. virsh: Print error if specified bandwidth is invalid for blockjob It's strange that the command fails but without any error if one specifies as not a number. 2011-08-22 KAMEZAWA Hiroyuki send-key: fix scan keycode map Now, bad key-code in send-key can cause segmentation fault in libvirt. (example) % virsh send-key --codeset win32 12 error: End of file while reading data: Input/output error This is caused by overrun at scanning keycode array. Fix it. 2011-08-22 Eric Blake build: work around older systemtap header Systemtap 1.2 tried to expand STAP_PROBE3 into an initialization: volatile __typeof__(arg) foo = arg; but that fails if arg was declared as 'char arg[100]'. Rather than make all callers to PROBE deal with the stupidity of , we instead make PROBE cast away the problem. Some of this preprocessor abuse copies ideas in src/libvirt.c. * daemon/libvirtd.h (PROBE): Add casts to all arguments, using... (VIR_ADD_CASTS, VIR_ADD_CAST, VIR_ADD_CAST2, VIR_ADD_CAST3) (VIR_ADD_CAST_EXPAND, VIR_ADD_CAST_PASTE, VIR_COUNT_ARGS) (VIR_ARG5, PROBE_EXPAND): New macros. Reported by Wen Congyang. 2011-08-19 Eric Blake virsh: properly interleave shared stdout and stderr Without this patch, invoking 'virsh >file 2>&1' results in error messages appearing before normal output, even if they occurred later in time than the normal output (since stderr is unbuffered, but stdout waits until a full buffer). * tools/virsh.c (print_job_progress, vshError): Flush between stream transitions. * tests/undefine: Test it. 2011-08-19 Eric Blake maint: simplify lots of libxml2 clients Repetitive patterns should be factored. The sign of a good factorization is a change that kills 5x more lines than it adds :) * src/conf/domain_conf.c (virDomainDeviceDefParse) (virDomainSnapshotDefParseString): Use new convenience macros. * src/conf/storage_conf.c (virStoragePoolDefParseSourceString): Likewise. * src/cpu/cpu.c (cpuCompareXML, cpuBaselineXML): Likewise. * src/esx/esx_vi.c (esxVI_Context_Execute): Likewise. * src/qemu/qemu_migration.c (qemuMigrationCookieXMLParseStr): Likewise. * src/security/virt-aa-helper.c (caps_mockup): Likewise. * src/test/test_driver.c (testOpenFromFile): Likewise. * tests/cputest.c (cpuTestLoadXML, cpuTestLoadMultiXML): Likewise. * tools/virsh.c (cmdFreecell, makeCloneXML, cmdVNCDisplay) (cmdTTYConsole, cmdDetachInterface, cmdDetachDisk) (cmdSnapshotCreate, cmdSnapshotCreateAs, cmdSnapshotCurrent) (cmdSnapshotList, cmdSnapshotParent): Likewise. 2011-08-19 Eric Blake xml: add another convenience function Often, we want to use XPath functions on the just-parsed document; fold this into the parser function for convenience. * src/util/xml.h (virXMLParseHelper): Add argument. (virXMLParseStrHelper, virXMLParseFileHelper): Delete. (virXMLParseCtxt, virXMLParseStringCtxt, virXMLParseFileCtxt): New macros. * src/libvirt_private.syms (xml.h): Remove deleted functions. * src/util/xml.c (virXMLParseHelper): Add argument. (virXMLParseStrHelper, virXMLParseFileHelper): Delete. 2011-08-19 Eric Blake maint: treat more libxml2 functions as free-like * cfg.mk (useless_free_options): Add xmlFreeDoc, xmlBufferFree. * src/esx/esx_vi.c (ESX_VI__TEMPLATE__FREE): Fix offenders. * tools/virsh.c (cmdFreecell, cmdVNCDisplay, cmdTTYConsole) (cmdDetachInterface, cmdDetachDisk, cmdSnapshotCreate) (cmdSnapshotCreateAs, cmdSnapshotList, cmdSnapshotParent): Likewise. 2011-08-19 Eric Blake test: rewrite test to match change in behavior Test failure exposed in commit 7d3390f. * tests/undefine: Fix to match updated test driver semantics. 2011-08-19 Eric Blake build: fix typo in recent test patch * src/test/test_driver.c (testDomainUndefineFlags): Use right variable name. 2011-08-19 Eric Blake Revert "xen: Allow to undefine a running domain (xm_internal)" ACK was given too soon. According to the code, the xm driver is only used for inactive domains, and has no notion of an active domain, thus, it cannot support undefine of a running domain. The real fix for xen needs to be in the unified driver and/or the xend level. This reverts commit 49186deda62161599877b0de6f7f4345c50d842a. 2011-08-19 Osier Yang xen: Allow to undefine a running domain (xm_internal) vmware: Allow to undefine a running domain uml: Allow to undefine a running domain test: Allow to undefine a running domain qemu: Allow to undefine a running domain openvz: Allow to undefine a running domain lxc: Allow to undefine a running domain libxl: Allow to undefine a running domain. Undefining a running domain will convert it to trasient, but keep the domain still running. qemu: Get memory balloon info correctly for text monitor * src/qemu/qemu_monitor_text.c: BALLOON_PREFIX was defined as "balloon: actual=", which cause "actual=" is stripped early before the real parsing. This patch changes BALLOON_PREFIX into "balloon: ", and modifies related functions, also renames "qemuMonitorParseExtraBalloonInfo" to "qemuMonitorParseBalloonInfo", as after the changing, it parses all the info returned by "info balloon". 2011-08-19 Eric Blake build: fix compilation on mingw64 * .gnulib: Update to latest, for getcwd fixes. * bootstrap: Resync to gnulib. 2011-08-19 Michal Privoznik storage: Flush host cache after write Although we are flushing cache after some critical writes (e.g. volume creation), after some others we do not (e.g. volume cloning). This patch fix this issue. That is for volume cloning, writing header of logical volume, and storage wipe. 2011-08-19 Michal Privoznik daemon: initialize GnuTLS When spice_tls is set but listen_tls is not, we don't initialize GnuTLS library. So any later gnutls call (e.g. during migration, where we initialize a certificate) will access uninitialized GnuTLS internal structs and throws an error. Although, we might now initialize GnuTLS twice, it is safe according to the documentation: This function can be called many times, but will only do something the first time. This patch creates 2 functions: virNetTLSInit and virNetTLSDeinit with respect to written above. 2011-08-19 Taku Izumi schedinfo: add missing documentation This patch adds the missing documentation about the scheduler parameter "vcpu_period" and "vcpu_quota". 2011-08-19 Eric Blake maint: ignore generated file I did 'git add .' while in the middle of 'make syntax-check', and it picked up a temporary file that should not be committed. * .gitignore: Ignore sc_* from syntax check. 2011-08-17 Eric Blake build: fix virnetsocket on mingw Regression introduced in commit 5d30db0. * src/rpc/virnetsocket.c (virNetSocketNewListenUNIX) [WIN32]: Use correct signature. 2011-08-17 Eric Blake build: fix virpidfile on mingw Regression introduced in commit b7e5ca4. Mingw lacks kill(), but we were only using it for a sanity check; so we can go with one less check. Also, on OOM error, this function should outright fail rather than claim that the pid file was successfully read. * src/util/virpidfile.c (virPidFileReadPathIfAlive): Skip kill call where unsupported, and report error on OOM. 2011-08-17 Daniel P. Berrange Ensure async packets never get marked for sync replies If a client had initiated a stream abort, it will have a call waiting for a reply in the queue. If more data continues to arrive on the stream, the abort command could mistakenly get signalled as complete. Remove the code from async data processing that looked for waiting calls. Add a sanity check to ensure no async call can ever be marked as needing a reply * src/rpc/virnetclient.c: Ensure async data packets can't trigger a reply 2011-08-17 Daniel P. Berrange Don't attempt to read from a stream if it is closed The I/O event callback processes incoming packets first, and then does outgoing packets. If the incoming packet caused the stream to close, then the attempt to process outgoing data resulted in an error. This caused libvirt to then send an error back to the client, but the stream had already been stopped. This confused the client since it sees 2 error events. * daemon/stream.c: Don't attempt read if stream is closed 2011-08-17 Tom Vijlbrief qemu: disk migration verbose progress A virsh command like: migrate --live --copy-storage-all Guest qemu+ssh://user@host/system --persistent --verbose shows Migration: [ 0 %] during the storage copy and does not start counting until the ram transfer starts Fix this by scraping optional disk transfer status, and adding it into the progress meter. 2011-08-17 Eric Blake virsh: concatenate qemu-monitor-command arguments Call me lazy, but: virsh qemu-monitor-command dom --hmp info status is nicer than: virsh qemu-monitor-command dom --hmp 'info status' * tools/virsh.c (cmdQemuMonitorCommand): Allow multiple arguments, for convenience. 2011-08-17 Osier Yang qemu: Init reattaching related members of pciDevice before reattach Otherwise the device will still be bound to pci-stub driver even it's set as "managed=yes" when do detaching. Of course, it won't triger any driver reprobing too. 2011-08-17 Eric Blake virsh: tweak misleading wording Fixes confusing docs introduced in commit 98369d3. * tools/virsh.c (cmdSnapshotParent): Operates on named snapshot, not current. 2011-08-16 Daniel P. Berrange Tweak debugging message in RPC client code Make it clearer that a log message is for an outgoing message 2011-08-16 Daniel P. Berrange Fix race condition in abort of stream If a stream gets a server initiated abort, the client may still send an abort request before it receives the server side abort. This causes the server to send back another abort for the stream. Since the protocol defines that abort is the last thing to be sent, the client gets confused by this second abort from the server. If the stream is already shutdown, just drop any client requested abort, rather than sending back another message. This fixes the regression from previous versions. Tested as follows In one virsh session virsh # start foo virsh # console foo In other virsh session virsh # destroy foo The first virsh session should be able to continue issuing commands without error. Prior to this patch it saw virsh # list error: Failed to list active domains error: An error occurred, but the cause is unknown virsh # list error: Failed to list active domains error: no call waiting for reply with prog 536903814 vers 1 serial 9 * src/rpc/virnetserverprogram.c: Drop abort requests for streams which no longer exist 2011-08-16 Daniel P. Berrange Ensure client streams are closed when marking a client for close Every active stream results in a reference being held on the virNetServerClientPtr object. This meant that if a client quit with any streams active, although all I/O was stopped the virNetServerClientPtr object would leak. This causes libvirtd to leak any file handles associated with open streams when a client quit To fix this, when we call virNetServerClientClose there is a callback invoked which lets the daemon release the streams and thus the extra references * daemon/remote.c: Add a hook to close all streams * daemon/stream.c, daemon/stream.h: Add API for releasing all streams * src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h: Allow registration of a hook to trigger when closing client 2011-08-16 Daniel P. Berrange Ensure stream is aborted when exiting console After running 'virsh console' in interactive mode, there was a missing call to virStreamAbort, which meant the server kept the stream resources open * tools/console.c: Abort stream when exiting 2011-08-16 Stefan Berger maint: fix some compilation issues on non-linux platforms (part 2) Get rid of the #if __linux__ check in virPidFileReadPathIfAlive that was preventing a check of a symbolic link in /proc//exe on non-linux platforms against an expected executable. Replace this with a run-time check testing whether the /proc//exe is a symbolic link and if so call the function doing the comparison against the expected file the link is supposed to point to. maint: fix some compilation issues on non-linux platforms This patch fixes *some* compilation issues on non-Linux platforms (cygwin). 2011-08-16 Peter Krempa daemon: Add early libvirtd start verbose errors. Early errors during start of libvirtd didn't have an error reporting mechanism and caused libvirtd to exit silently (only the return value indicated an error). Libvirt logging is initialized very early using enviroment variables and the internal error reporting API is used to report early errors. v2 changes: - print errors unconditionaly before logging starts - fix message to US spelling v2.5 changes: - initialize logging from enviroment - log all early errors using VIR_ERROR v3 changes: - move virSetLogFromEnv() after virInitialize() fixes: https://bugzilla.redhat.com/show_bug.cgi?id=728654 2011-08-16 Stefan Berger maint: add missing symbols Add missing symbols to libvirt_private.syms. 2011-08-16 Roopa Prabhu macvtap: Fix getPhysfn to get the PF of a direct attach network interface This patch renames getPhysfn to getPhysfnDev and adds code to get the Physical function and Virtual Function index of the direct attach linkdev (if the direct attach interface is a SRIOV VF). The idea is to send the port profile message to a PF if the direct attach interface is a SRIOV VF. interface: Add functions to get sriov PF/VF relationship of a net interface This patch adds the following functions to get PF/VF relationship of an SRIOV network interface: ifaceIsVirtualFunction: Function to check if a network interface is a SRIOV VF ifaceGetVirtualFunctionIndex: Function to get VF index if a network interface is a SRIOV VF ifaceGetPhysicalFunction: Function to get the PF net interface name of a SRIOV VF net interface pci: Add helper functions for sriov devices This patch adds the following helper functions: pciDeviceIsVirtualFunction: Function to check if a pci device is a sriov VF pciGetVirtualFunctionIndex: Function to get the VF index of a sriov VF pciDeviceNetName: Function to get the network device name of a pci device pciConfigAddressCompare: Function to compare pci config addresses pci: Move some pci sriov helper code out of node device driver to util/pci This patch moves some of the sriov related pci code from node_device driver to src/util/pci.[ch]. Some functions had to go thru name and argument list change to accommodate the move. 2011-08-15 Eric Blake maint: ignore generated file * .gitignore: Ignore qemu-sanlock.conf. 2011-08-15 Laine Stump qemu: support event_idx parameter for virtio disk and net devices In some versions of qemu, both virtio-blk-pci and virtio-net-pci devices can have an event_idx setting that determines some details of event processing. When it is enabled, it "reduces the number of interrupts and exits for the guest". qemu will automatically enable this feature when it is available, but there may be cases where this new feature could actually make performance worse (NB: no such case has been found so far). As a safety switch in case such a situation is encountered in the field, this patch adds a new attribute "event_idx" to the element of both disk and interface devices. event_idx can be set to "on" (to force event_idx on in case qemu has it disabled by default) or "off" (for force event_idx off). In the case that event_idx support isn't present in qemu, the attribute is ignored (this on the advice of the qemu developer). docs/formatdomain.html.in: document the new flag (marking it as "don't mess with this!" docs/schemas/domain.rng: add event_idx in appropriate places src/conf/domain_conf.[ch]: add event_idx to parser and formatter src/libvirt_private.syms: export virDomainVirtioEventIdx(From|To)String src/qemu/qemu_capabilities.[ch]: detect and report event_idx in disk/net src/qemu/qemu_command.c: add event_idx parameter to qemu commandline when appropriate. tests/qemuxml2argvdata/qemuxml2argv-event_idx.args, tests/qemuxml2argvdata/qemuxml2argv-event_idx.xml, tests/qemuxml2argvtest.c, tests/qemuxml2xmltest.c: test cases for event_idx. 2011-08-15 Peter Krempa qemu: error message should show uri instead of (null) Fix pointer for error message uri if domain migration fails. BZ# 730244 2011-08-15 Jiri Denemark qemu: Use fd: protocol for migration By opening a connection to remote qemu process ourselves and passing the socket to qemu we get much better errors than just "migration failed" when the connection is opened by qemu. qemu: Use virNetSocket for tunneled migration qemu: Refactor do{Tunnel,Native}Migrate functions The core of these two functions is very similar and most of it is even exactly the same. Factor out the core functionality into a separate function to remove code duplication and make further changes easier. Support changing UNIX socket owner in virNetSocketNewListenUNIX This patch allows owner's UID to be changed as well. Add backlog parameter to virNetSocketListen So that callers can change the default value. 2011-08-15 Daniel P. Berrange Add API for duplicating a socket/client file descriptor * src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virNetSocketDupFD() * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add virNetClientDupFD() and virNetClientGetFD() 2011-08-15 Douglas Schilling Landgraf libvirtd.init.in: stop/restart() - wrong return value in case of failure The function stop() was always returning 0 (OK) from killproc() even in case of error. 2011-08-15 Osier Yang daemon: Fix regression of libvirtd reloading support This is introduced by commit df0b57a95a, which forgot to add signal handler for SIGHUP. A simple reproduce method: 1) Create a domain XML under /etc/libvirt/qemu 2) % kill -SIGHUP $(pidof libvirtd) 3) % virsh list --all (the new created domain XML is not listed) 2011-08-13 Eric Blake maint: let emacs avoid tabs in rng files * .dir-locals.el: Add nxml-mode preferences. 2011-08-13 Philipp Hahn Swap virDomain / virFomainSnapshot declaration In preparation for storing the domain description with the snapshot, swap the order of declaration. 2011-08-13 Eric Blake build: fix recent build failures With gcc 4.5.1: util/virpidfile.c: In function 'virPidFileAcquirePath': util/virpidfile.c:308:66: error: nested extern declaration of '_gl_verify_function2' [-Wnested-externs] Then in tests/commandtest.c, the new virPidFile APIs need to be used. * src/util/virpidfile.c (virPidFileAcquirePath): Move verify to top level. * tests/commandtest.c: Use new pid APIs. 2011-08-12 Daniel P. Berrange Convert libvirtd to use crash-safe pidfile APIs Remove the current libvirtd pidfile handling code, in favour of calling out to the new APIs. This ensures libvirtd's pidfile handling is crashsafe This also means that the non-root libvirtd instances (for handling qemu:///session URIs) can now safely use pidfiles without racing * daemon/libvirtd.c: Switch to use virPidFileAcquire and virPidFileRelease 2011-08-12 Daniel P. Berrange Add some APIs which use locking for crashsafe pidfile handling In daemons using pidfiles to protect against concurrent execution there is a possibility that a crash may leave a stale pidfile on disk, which then prevents later restart of the daemon. To avoid this problem, introduce a pair of APIs which make use of virFileLock to ensure crash-safe & race condition-safe pidfile acquisition & releae * src/libvirt_private.syms, src/util/virpidfile.c, src/util/virpidfile.h: Add virPidFileAcquire and virPidFileRelease 2011-08-12 Daniel P. Berrange Introduce functions for checking whether a pidfile is valid In some cases the caller of virPidFileRead might like extra checks to determine whether the pid just read is really the one they are expecting. This adds virPidFileReadIfAlive which will check whether the pid is still alive with kill(0, -1), and (on linux only) will look at /proc/$PID/path * libvirt_private.syms, util/virpidfile.c, util/virpidfile.h: Add virPidFileReadIfValid and virPidFileReadPathIfValid * network/bridge_driver.c: Use new APIs to check PID validity 2011-08-12 Daniel P. Berrange Move pidfile functions into util/virpidfile.{c,h} The functions for manipulating pidfiles are in util/util.{c,h}. We will shortly be adding some further pidfile related functions. To avoid further growing util.c, this moves the pidfile related functions into a dedicated virpidfile.{c,h}. The functions are also all renamed to have 'virPidFile' as their name prefix * util/util.h, util/util.c: Remove all pidfile code * util/virpidfile.c, util/virpidfile.h: Add new APIs for pidfile handling. * lxc/lxc_controller.c, lxc/lxc_driver.c, network/bridge_driver.c, qemu/qemu_process.c: Add virpidfile.h include and adapt for API renames 2011-08-12 Daniel P. Berrange Add virFileLock and virFileUnlock APIs Add some simple wrappers around the fcntl() discretionary file locking capability. * src/util/util.c, src/util/util.h, src/libvirt_private.syms: Add virFileLock and virFileUnlock APIs 2011-08-12 Eric Blake virsh: fix dead store Two copy-and-paste bugs in a row. :( * tools/virsh.c (cmdUndefine): Also avoid dead store. 2011-08-12 Eric Blake virsh: fix logic bug Bug introduced in commit 9a0ec36. * tools/virsh.c (cmdUndefine): Add missing line. 2011-08-12 Eric Blake vbox, xenapi: add virDomainUndefineFlags We forgot to add virDomainUndefineFlags for a couple of hypervisors. This wires up trivial versions (since neither hypervisor supports managed save yet, they do not need to support any flags). * src/vbox/vbox_tmpl.c (vboxDomainCreateXML): Update caller. (vboxDomainUndefine): Move guts... (vboxDomainUndefineFlags): ...to new function. * src/xenapi/xenapi_driver.c (xenapiDomainUndefine) (xenapiDomainUndefineFlags): Likewise. 2011-08-12 Eric Blake virsh: don't reject undefine on active domain The public API documents that undefine may be used to transition a running persistent domain into a transient one. Many drivers still do not support this usage, but virsh shouldn't be getting in the way of those that do support it. This also drops a redundant conditional; vshCommandOptString guaranteed that name was non-NULL. * tools/virsh.c (cmdUndefine): Allow undefine on active domains; the drivers may still reject it, but it is a valid API usage. * tests/undefine (error): Fix the test to match. 2011-08-12 Philipp Hahn Improve tokenizing of linkable terms Currently only tabs and blanks are used for tokenizing the description, which breaks when a term is at the end of a line or has () appended to it. 1. Use also other white space characters such as new-lines and carriage return for splitting. 2. Remove some common non-word characters from the token before lookup. 2011-08-12 Daniel P. Berrange Add some docs about the RPC protocol and APIs * remote.html.in: Remove obsolete notes about internals of the RPC protocol * internals/rpc.html.in: Extensive docs on RPC protocol/API * sitemap.html.in: Add new page 2011-08-11 Philipp Hahn Align table cells on top When the description of an entry is too long and needs multiple lines, all other table cells of the same row are currently vertically aligned on center. Without row borders or different background colors for alternating rows this is hard to read. Change the style-sheet to align the table cells of a row on top. 2011-08-11 Peter Krempa virsh: Add dir type for listing volumes with vol-list Fix of output of detailed volume list. BZ #727088 2011-08-11 Peter Krempa storage: Directory shouldn't be listed as type 'file' Fix internals of libvirt for new storage volume type. Libvirt reported an invalid type of the volume. BZ #727088 2011-08-11 Eric Blake virsh: add snapshot-parent Down the road, I want to add virDomainSnapshotGetParent, and use the new API rather than xml scraping; but this virsh command can be implemented even without the new API. * tools/virsh.c (cmdSnapshotParent): New command. * tools/virsh.pod (snapshot-parent): Document it. 2011-08-11 Eric Blake qemu: silence clang false positive Our logic throws off analyzer tools: ptr var = NULL; if (flags == 0) flags = live ? _LIVE : _CONFIG; if (flags & _LIVE) do stuff if (flags & _CONFIG) var = non-null; if (flags & _LIVE) do more stuff else if (flags & _CONFIG) use var the tools keep thinking that var can still be NULL in the last if clause, adding the hint shuts them up. * src/qemu/qemu_driver.c (qemuDomainSetBlkioParameters): Add a static analysis hint. 2011-08-11 Laine Stump network: eliminate potential memory leak on parse failure While the first encountered dns host record is being parsed, it's possible for virNetworkDef::hosts to point to memory that has been allocated, but virNetworkDef::nhosts to still be 0. If there is a failure during that time, virNetworkDef::hosts will be leaked. Although this isn't currently the case for virNetworkDef::txtrecords, it could become that way through future re-factoring, and it hurts nothing to restructure the freeing of txtrecord data to match that of hosts data. 2011-08-11 Stefan Berger nwfilter: tolerate disappearing interfaces while instantiating filter When instantiating a filter, a VM may disappear and remove its tap interfaces. Tolerate this case and don't report an error. 2011-08-11 Cole Robinson qemu: Fix -chardev udp if parameters are omitted The following XML: is accepted by domain_conf.c but maps to the qemu command line: -chardev udp,host=127.0.0.1,port=2222,localaddr=(null),localport=(null) qemu can cope with everything omitting except the connection port, which seems to also be the intent of domain_conf validation, so let's not generate bogus command lines for that case. The defaults are empty strings for addresses and 0 for the localport Additionally, tweak the qemu cli parsing to handle omitted host parameters for -serial udp 2011-08-11 Laine Stump docs: use IPv6 addresses in range reserved for documentation Someone in an IRC channel or an email pointed out a few days ago that the examples of IPv6 addresses in the libvirt documentation were not in the officially reserved "documentation" range. This addresses their concern. 2011-08-11 Eric Blake virsh: add virsh snapshot-current --name Sometimes, full XML is too much; since most snapshot commands operate on a snapshot name, there should be an easy way to get at the current snapshot's name. For example: virsh snapshot-revert dom `virsh snapshot-current dom --name` * tools/virsh.c (cmdSnapshotCurrent): Add an option. * tools/virsh.pod (snapshot-current): Document it. 2011-08-11 Eric Blake virsh: fix snapshot-create-as to handle arbitrary names Found this working on the next patch to use xpath to parse arbitrary names back out. * tools/virsh.c (cmdSnapshotCreateAs): Escape user input. 2011-08-11 Eric Blake managedsave: prohibit use on transient domains Transient domains reject attempts to set autostart, and using virDomainCreate to restart a domain only works on persistent domains. Therefore, managed save makes no sense on transient domains, and should be rejected up front rather than creating an otherwise unrecoverable managed save file. Besides, transient domains imply that a lot more management is being done by the upper layer; this includes the assumption that the upper layer is okay managing the saved state file created by virDomainSave, and does not need to use managed save. * src/libvirt.c: Document that transient domains are incompatible with managed save. * src/qemu/qemu_driver.c (qemuDomainManagedSave): Enforce it. * src/libxl/libxl_driver.c (libxlDomainManagedSave): Likewise. 2011-08-10 Laine Stump docs: describe new virtual switch configuration in network XML docs This should have been done with the rest of the patch for virtual switch / network device abstraction. If documents the new elements (and new usage of existing elements) in the XML to support libvirt networks that use existing host bridges and macvtap direct connections, as well as the new element. 2011-08-10 Juerg Haefliger Bugfix: Check stdoutWatch before removing the handler. 2011-08-10 Eric Blake qemu: minor formatting cleanup I noticed some inconsistent use of 'else'. * src/qemu/qemu_driver.c (qemuCPUCompare) (qemuDomainSnapshotCreateXML, qemuDomainRevertToSnapshot) (qemuDomainSnapshotDiscard): Match coding conventions. 2011-08-09 Philipp Hahn Fix memory leak while scanning snapshots If a snapshot with the name already exists, virDomainSnapshotAssignDef() just returns NULL, in which case the snapshot definition is leaked. Currently this leak is not a big problem, since qemuDomainSnapshotLoad() is only called once during initial startup of libvirtd. 2011-08-08 Eric Blake build: fix regression in large file support * .gnulib: Update to latest, for largefile fix. 2011-08-08 Daniel P. Berrange Allow use of file images for LXC container filesystems A previous commit gave the LXC driver the ability to mount block devices for the container filesystem. Through use of the loopback device functionality, we can build on this to support use of plain file images for LXC filesytems. By setting the LO_FLAGS_AUTOCLEAR flag we can ensure that the loop device automatically disappears when the container dies / shuts down * src/lxc/lxc_container.c: Raise error if we see a file based filesystem, since it should have been turned into a loopback device already * src/lxc/lxc_controller.c: Rewrite any filesystems of type=file, into type=block, by binding the file image to a free loop device 2011-08-08 Daniel P. Berrange Fix typo in LXC cgroups setup error message * src/lxc/lxc_controller.c: s/PYT/PTY/ 2011-08-08 Daniel P. Berrange Allow use of block devices for guest filesystem Currently the LXC driver can only populate filesystems from host filesystems, using bind mounts. This patch allows host block devices to be mounted. It autodetects the filesystem format at mount time, and adds the block device to the cgroups ACL. Example usage is * src/lxc/lxc_container.c: Mount block device filesystems * src/lxc/lxc_controller.c: Add block device filesystems to cgroups ACL 2011-08-08 Daniel P. Berrange Don't mount /dev for application containers An application container shouldn't get a private /dev. Fix the regression from 6d37888e6a35a37e6faf7c0a1b1b4d9a5dee1285 * src/lxc/lxc_container.c: Don't mount /dev for app containers 2011-08-05 Jiri Denemark qemu: Avoid overwriting errors from virGetHostname 2011-08-04 Eric Blake qemu: avoid crash on process attach Detected by ccc-analyzer, reported by Alex Jia. qemuProcessStart always calls qemuProcessWaitForMonitor with a non-negative position, but qemuProcessAttach always calls with -1. In the latter case, there is no log file we can scrape, so we also should not be trying to scrape the logs if the qemu process died at the very end. * src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Don't try to read from log in qemuProcessAttach case. 2011-08-04 Laine Stump network: eliminate lag in updating dnsmasq hosts files This addresses https://bugzilla.redhat.com/show_bug.cgi?id=713728 When "defining" a new network (or one that exists but isn't currently active) the new definition is stored in network->def, but for a network that already exists and is active, the new definition is stored in network->newDef, and then moved over to network->def as soon as the network is destroyed. However, the code that writes the dhcp and dns hosts files used by dnsmasq was always using network->def for its information, even when the new data was actually in network->newDef, so the hosts files always lagged one edit behind the definition. This patch changes the code to keep the pointer to the new definition after it's been assigned into the network, and use it directly (regardless of whether it's stored in network->newDef or network->def) to construct the hosts files. 2011-08-04 Alex Jia qemu: avoid dead store in qemuMonitorTextBlockJob Value stored to 'ret' is never read, so remove this dead assignment. * src/qemu/qemu_monitor_text.c: kill dead assignment. 2011-08-04 Alex Jia qemu: avoid dead store in qemuProcessStart Value stored to 'ret' is never read, in fact, 'cleanup' section will directly return -1 when function is fail, so remove this dead assignment. * src/qemu/qemu_process.c: kill dead assignment. 2011-08-04 Guannan Ren rpc:fix sasl session relocking intead of unlocking it When trying to use any SASL authentication for TCP sockets by setting auth_tls = "sasl" in libvirtd.conf on server side, the client will hang because of the sasl session relocking other than dropping the lock when exiting virNetSASLSessionExtKeySize() * src/rpc/virnetsaslcontext.c: virNetSASLSessionExtKeySize drop the lock on exit 2011-08-04 Osier Yang daemon: Unlink unix socket paths on shutdown This patch introduces a internal RPC API "virNetServerClose", which is standalone with "virNetServerFree". it closes all the socket fds, and unlinks the unix socket paths, regardless of whether the socket is still referenced or not. This is to address regression bug: https://bugzilla.redhat.com/show_bug.cgi?id=725702 2011-08-03 Eric Blake build: update to latest gnulib I noticed that with 0.9.4, gnulib ended up replacing pthread_sigmask on glibc, even though glibc's works perfectly fine. It turns out to have been an upstream gnulib bug. * .gnulib: Update to latest, for pthread_sigmask fix. 2011-08-03 Matthias Bolte Fix detection of GnuTLS 1.x.y Detection based on gnutls_session doesn't work because GnuTLS 2.x.y comes with a compat.h that defines gnutls_session to gnutls_session_t. Instead detect this based on LIBGNUTLS_VERSION_MAJOR. Move this from configure/config.h to gnutls_1_0_compat.h and make sure that all users include gnutls_1_0_compat.h properly. Also fix header guard in gnutls_1_0_compat.h. 2011-08-03 Daniel Veillard Release of libvirt-0.9.4 * configure.ac docs/news.html.in libvirt.spec.in: updates for new release * po/*.po*: pulled translations from the transifex teams and regenerated localizations 2011-08-03 Eric Blake qemu: silence coverity false positives Coverity gets confused by our logic. Add some hints to silence false positives. * src/qemu/qemu_driver.c (qemudDomainGetVcpuPinInfo): Add hint. (qemuDomainGetMemoryParameters): Likewise. 2011-08-03 Eric Blake util: plug memory leak Leak detected by Coverity; only possible on unlikely ptsname_r failure. Additionally, the man page for ptsname_r states that failure is merely non-zero, not necessarily -1. * src/util/util.c (virFileOpenTtyAt): Avoid leak on ptsname_r failure. 2011-08-03 Eric Blake interface: drop dead code Coverity detected that ifaceGetNthParent had already dereferenced 'nth' prior to the conditional; all callers already complied with passing a non-NULL pointer so make this part of the contract. * src/util/interface.h (ifaceGetNthParent): Add annotations. * src/util/interface.c (ifaceGetNthParent): Drop useless null check. 2011-08-03 Eric Blake rpc: silence coverity false positives In virNetServerNew, Coverity didn't realize that srv->mdsnGroupName can only be non-NULL if mdsnGroupName was non-NULL. In virNetServerRun, Coverity didn't realize that the array is non-NULL if the array count is non-zero. * src/rpc/virnetserver.c (virNetServerNew): Use alternate pointer. (virNetServerRun): Give coverity a hint. 2011-08-03 Eric Blake rpc: avoid null deref Detected by Coverity. * src/rpc/virnetserverclient.c (virNetServerClientDispatchRead): Avoid null deref on OOM. 2011-08-03 Eric Blake build: silence coverity false positives Coverity complained that 395 out of 409 virAsprintf calls are checked, and therefore assumed that the remaining cases are bugs waiting to happen. But in each of these cases, a failed virAsprintf will properly set the target string to NULL, and pass on that failure to the caller, without wasting efforts to check the call. Adding the ignore_value silences Coverity. * src/conf/domain_audit.c (virDomainAuditGetRdev): Ignore virAsprintf return value, when it behaves like we need. * src/network/bridge_driver.c (networkDnsmasqLeaseFileNameDefault) (networkRadvdConfigFileName, networkBridgeDummyNicName) (networkRadvdPidfileBasename): Likewise. * src/util/storage_file.c (absolutePathFromBaseFile): Likewise. * src/openvz/openvz_driver.c (openvzGenerateContainerVethName): Likewise. * src/util/command.c (virCommandTranslateStatus): Likewise. 2011-08-03 Eric Blake qemu: avoid memory leaks Quite a few leaks detected by coverity. For chr, the leaks were close enough to the allocations to plug in place; for disk, the leaks were separated from the allocation by enough other lines with intermediate failure cases that I refactored the cleanup instead. * src/qemu/qemu_command.c (qemuParseCommandLine): Plug leaks. 2011-08-02 Eric Blake qemu: plug child process leak on domain core dump Detected by Coverity. Leak introduced by typo in commit 58e668d2. * src/qemu/qemu_driver.c (doCoreDump): Use correct function. 2011-08-02 Eric Blake qemu: remove dead code Warning detected by Coverity. No need for the NULL check, and removing it silences the warning without any semantic change. * src/qemu/qemu_migration.c (qemuMigrationFinish): All entries to endjob had non-NULL vm. 2011-08-02 Eric Blake conf: avoid memory leak on disk operations Detected by Coverity. Leak on meta introduced in commit 85aa40e. * src/conf/domain_conf.c (virDomainDiskDefForeachPath): Plug leak. 2011-08-02 Eric Blake rpc: avoid crash on error Detected by Coverity. Freeing the wrong variable results in both a memory leak and the likelihood of the caller dereferencing through a freed pointer. * src/rpc/virnettlscontext.c (virNetTLSSessionNew): Free correct variable. 2011-08-02 Eric Blake qemu: avoid null deref on block pull error Coverity detected that 5 of 6 callers of virJSONValueArrayGet checked for a NULL return; and that by not checking we risk a null deref during an error. The error is unlikely since the prior call to virJSONValueArraySize would probably have already caught any botched JSON array parse, but better safe than sorry. * src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetBlockJobInfo): Check for NULL. (qemuMonitorJSONExtractPtyPaths): Fix typo. 2011-08-02 Eric Blake rpc: correctly process sasl whitelist globs Detected by Coverity. We want to compare the result of fnmatch 'rv', not our pre-set return value 'ret'. * src/rpc/virnetsaslcontext.c (virNetSASLContextCheckIdentity): Check correct variable. 2011-08-02 Eric Blake build: allow caching the input to STATIC_ANALYSIS Right now, every re-run of configure re-evaluates whether a static analysis tool is in use. But if you run configure under coverity, make a tweak, and then do an incremental rebuild with gcc but not coverity to test the tweak, then rerun a build under coverity, then configure does not get rerun, and static analysis ends up with lots of false positives. This patch caches the static analysis result, and also makes it easier to force static analysis even if the existing checks are insufficient to detect newer versions of the static analyzer tools. * configure.ac (lv_cv_static_analysis): New cache variable. 2011-08-02 Eric Blake fdstream: drop delete argument Revert 6a1f5f568f8. Now that libvirt_iohelper takes fds by inheritance rather than by open() (commit 1eb66479), there is no longer a race where the parent can unlink() a file prior to the iohelper open()ing the same file. From there, it makes more sense to have the callers both create and unlink, rather than the caller create and the stream unlink, since the latter was only needed when iohelper had to do the unlink. * src/fdstream.h (virFDStreamOpenFile, virFDStreamCreateFile): Callers are responsible for deletion. * src/fdstream.c (virFDStreamOpenFileInternal): Don't leak created file on failure. (virFDStreamOpenFile, virFDStreamCreateFile): Drop parameter. * src/lxc/lxc_driver.c (lxcDomainOpenConsole): Update callers. * src/qemu/qemu_driver.c (qemuDomainScreenshot) (qemuDomainOpenConsole): Likewise. * src/storage/storage_driver.c (storageVolumeDownload) (storageVolumeUpload): Likewise. * src/uml/uml_driver.c (umlDomainOpenConsole): Likewise. * src/vbox/vbox_tmpl.c (vboxDomainScreenshot): Likewise. * src/xen/xen_driver.c (xenUnifiedDomainOpenConsole): Likewise. 2011-08-02 Eric Blake screenshot: don't unlink bogus file The previous qemu patch could end up calling unlink(tmp) before tmp was the name of a valid file (unlinking a fileXXXXXX template instead), or calling unlink(tmp) twice on success (once here, and once at the end of the stream). Meanwhile, vbox also suffered from the same leaked tmp file bug. * src/qemu/qemu_driver.c (qemuDomainScreenshot): Don't unlink on success, or on invalid name. * src/vbox/vbox_tmpl.c (vboxDomainScreenshot): Don't leak temp file. 2011-08-02 Eric Blake python: avoid unlikely sign extension bug Detected by Coverity; same analysis as for commit f73198df. * python/libvirt-override.c (libvirt_virDomainGetVcpuPinInfo): Use correct type. 2011-08-02 Eric Blake rpc: avoid uninitialized memory use Spotted by Coverity. Gnutls documents that buffer must be NULL if gnutls_x509_crt_get_key_purpose_oid is to be used to determine the correct size needed for allocating a buffer. * src/rpc/virnettlscontext.c (virNetTLSContextCheckCertKeyPurpose): Initialize buffer. 2011-08-02 Eric Blake rpc: avoid double close on error Spotted by coverity. If pipe2 fails, then we attempt to close uninitialized fds, which may result in a double-close. * src/rpc/virnetserver.c (virNetServerSignalSetup): Initialize fds. 2011-08-02 Michal Privoznik qemu: Unlink temporary file on failure Although virFDStreamOpenFile will unlink it once opened, when we hit error path, we must unlink it by hand. 2011-08-02 Eric Blake doc: fix confusing statement about required privileges src/libvirt.c: "may requires" is confusing; the s was extra. 2011-08-02 Eric Blake rpc: avoid libvirtd crash on unexpected client close Steps to reproduce this problem (vm1 is not running): for i in `seq 50`; do virsh managedsave vm1& done; killall virsh Pre-patch, virNetServerClientClose could end up setting client->sock to NULL prior to other cleanup functions trying to use client->sock. This fixes things by checking for NULL in more places, and by deferring the cleanup until after all queued messages have been served. * src/rpc/virnetserverclient.c (virNetServerClientRegisterEvent) (virNetServerClientGetFD, virNetServerClientIsSecure) (virNetServerClientLocalAddrString) (virNetServerClientRemoteAddrString): Check for closed socket. (virNetServerClientClose): Rearrange close sequence. Analysis from Wen Congyang. 2011-08-02 Alex Jia virsh: fix memory leak in cmdNetworkInfo * tools/virsh.c: avoid memory leak in cmdNetworkInfo. * how to reproduce? % valgrind -v --leak-check=yes virsh net-info default https://bugzilla.redhat.com/show_bug.cgi?id=722806 2011-08-02 Alex Jia doc: fix incorrect option in blockjob * tools/virsh.pod: change 'live' to 'abort' based on actual option 2011-08-01 Jean-Baptiste Rouault openvz: detect when a domain was shut down from the inside This patch adds an internal function openvzGetVEStatus to get the real state of the domain. This function is used in various places in the driver, in particular to detect when the domain has been shut down by the user with the "halt" command. 2011-08-01 Eric Blake qemu: fix crash when mixing sync and async monitor jobs Currently, we attempt to run sync job and async job at the same time. It means that the monitor commands for two jobs can be run in any order. In the function qemuDomainObjEnterMonitorInternal(): if (priv->job.active == QEMU_JOB_NONE && priv->job.asyncJob) { if (qemuDomainObjBeginNestedJob(driver, obj) < 0) We check whether the caller is an async job by priv->job.active and priv->job.asynJob. But when an async job is running, and a sync job is also running at the time of the check, then priv->job.active is not QEMU_JOB_NONE. So we cannot check whether the caller is an async job in the function qemuDomainObjEnterMonitorInternal(), and must instead put the burden on the caller to tell us when an async command wants to do a nested job. Once the burden is on the caller, then only async monitor enters need to worry about whether the VM is still running; for sync monitor enter, the internal return is always 0, so lots of ignore_value can be dropped. * src/qemu/THREADS.txt: Reflect new rules. * src/qemu/qemu_domain.h (qemuDomainObjEnterMonitorAsync): New prototype. * src/qemu/qemu_process.h (qemuProcessStartCPUs) (qemuProcessStopCPUs): Add parameter. * src/qemu/qemu_migration.h (qemuMigrationToFile): Likewise. (qemuMigrationWaitForCompletion): Make static. * src/qemu/qemu_domain.c (qemuDomainObjEnterMonitorInternal): Add parameter. (qemuDomainObjEnterMonitorAsync): New function. (qemuDomainObjEnterMonitor, qemuDomainObjEnterMonitorWithDriver): Update callers. * src/qemu/qemu_driver.c (qemuDomainSaveInternal) (qemudDomainCoreDump, doCoreDump, processWatchdogEvent) (qemudDomainSuspend, qemudDomainResume, qemuDomainSaveImageStartVM) (qemuDomainSnapshotCreateActive, qemuDomainRevertToSnapshot): Likewise. * src/qemu/qemu_process.c (qemuProcessStopCPUs) (qemuProcessFakeReboot, qemuProcessRecoverMigration) (qemuProcessRecoverJob, qemuProcessStart): Likewise. * src/qemu/qemu_migration.c (qemuMigrationToFile) (qemuMigrationWaitForCompletion, qemuMigrationUpdateJobStatus) (qemuMigrationJobStart, qemuDomainMigrateGraphicsRelocate) (doNativeMigrate, doTunnelMigrate, qemuMigrationPerformJob) (qemuMigrationPerformPhase, qemuMigrationFinish) (qemuMigrationConfirm): Likewise. * src/qemu/qemu_hotplug.c: Drop unneeded ignore_value. 2011-08-01 Alex Jia qemu: fix return value issue whether or not previous return value is -1, the following codes will be executed for a inactive guest in src/qemu/qemu_driver.c: ret = virDomainSaveConfig(driver->configDir, persistentDef); and if everything is okay, 'ret' is assigned to 0, the previous 'ret' will be overwritten, this patch will fix this issue. * src/qemu/qemu_driver.c: avoid return value is overwritten when give a argument in out of blkio weight range for a inactive guest. * how to reproduce? % virsh blkiotune ${guestname} --weight 10 % echo $? Note: guest must be inactive, argument 10 in out of blkio weight range, and can get a error information by checking libvirtd.log, however, virsh hasn't raised any error information, and return value is 0. https://bugzilla.redhat.com/show_bug.cgi?id=726304 2011-08-01 Alex Jia qemu: fix return value issue in qemuDomainSetMemoryParameters whether or not previous return value is -1, the following codes will be executed for a inactive guest in qemuDomainSetMemoryParameters: ret = virDomainSaveConfig(driver->configDir, persistentDef); and if everything is okay, 'ret' is assigned to 0, the previous 'ret' will be overwritten, this patch will fix this issue. * src/qemu/qemu_driver.c: avoid return value is overwritten when set min_guarante value to a inactive guest. * how to reproduce? % virsh memtune ${guestname} --min_guarante 1024 % echo $? Note: guest must be inactive, in fact, 'min_guarante' hasn't been implemented in memory tunable, and I can get the error when check actual libvirtd.log, however, virsh hasn't raised any error information, and return value is 0. 2011-08-01 Matthias Bolte virsh: Fix vol-name and vol-pool commands This commands don't have a --pool option, so don't tell vshCommandOptVolBy that there could be one. This made vshCommandOptString for pooloptname fail and an "missing option" error was reported. Make pooloptname optional for vshCommandOptVolBy. 2011-08-01 Guannan Ren Correct the default value of lock_manager in qemu.conf * src/qemu/qemu.conf: switch back the comment for the default lock manager to sanlock 2011-08-01 Osier Yang qemu: Fix a regression of domjobabort Introduced by f9a837da73a11ef, the condition is not changed after the else clause is removed. So now it quit with "domain is not running" when the domain is running. However, when the domain is not running, it reports "no job is active". How to reproduce: 1) % virsh start $domain % virsh domjobabort $domain error: Requested operation is not valid: domain is not running 2) % virsh destroy $domain % virsh domjobabort $domain error: Requested operation is not valid: no job is active on the domain 3) % virsh save $domain /tmp/$domain.save Before above commands finished, try to abort job in another terminal % virsh domabortjob $domain error: Requested operation is not valid: domain is not running 2011-08-01 Matthias Bolte conf: Don't leak the virtual port profile in virNetworkDefFree Reported by Alex Jia. 2011-07-30 Eric Blake save: generate idempotent inactive xml for running domain Originally noticed by comparing the xml generated by virDomainSave with the xml produced by reparsing and redumping that xml, but I also did an audit of every last use of VIR_DOMAIN_XML_INACTIVE in domain_conf.c to ensure that no other discrepancies exist. * src/conf/domain_conf.c (virDomainDeviceInfoIsSet): Add parameter, and update all callers. Make static. (virDomainNetDefFormat): Skip generated ifname. (virDomainDefFormatInternal): Skip default . (virDomainChrSourceDefParseXML): Skip generated pty path, and add parameter. Update callers. * src/conf/domain_conf.h (virDomainDeviceInfoIsSet): Delete. * src/libvirt_private.syms (domain_conf.h): Update. 2011-07-30 Eric Blake conf: make 'vnet' prefix a macro Using a macro ensures that all the code is looking for the same prefix. * src/conf/domain_conf.h (VIR_NET_GENERATED_PREFIX): New macro. * src/conf/domain_conf.c (virDomainNetDefParseXML): Use it. * src/uml/uml_conf.c (umlConnectTapDevice): Likewise. * src/qemu/qemu_command.c (qemuNetworkIfaceConnect): Likewise. Suggested by Laine Stump. 2011-07-29 Laine Stump network: don't forward DNS requests from isolated networks This is in response to: https://bugzilla.redhat.com/show_bug.cgi?id=723862 which points out that a guest on an "isolated" network could potentially exploit the DNS forwarding provided by dnsmasq to create a communication channel to the outside. This patch eliminates that possibility by adding the "--no-resolv" argument to the dnsmasq commandline, which tells dnsmasq to not forward on any requests that it can't resolve itself (by looking at its own static hosts files and runtime list of dhcp clients), but to instead return a failure for those requests. This shouldn't cause any undesirable change from current behavior, even in the case where a guest is currently configured with multiple interfaces, one of them being connected to an isolated network, and another to a network that does have connectivity to the outside. If the isolated network's DNS server is queried for a name it doesn't know, it will return "Refused" rather than "Unknown", which indicates to the guest that it should query other servers, so it then queries the connected DNS server, and gets the desired response. 2011-07-29 Eric Blake build: fix include path for cygwin Without this, cygwin failed to compile: In file included from ../src/rpc/virnetmessage.h:24, from ../src/rpc/virnetclient.h:27, from remote/remote_driver.c:31: ../src/rpc/virnetprotocol.h:9:21: error: rpc/rpc.h: No such file or directory With that fixed, compilation warned: rpc/virnetsocket.c: In function 'virNetSocketNewListenUNIX': rpc/virnetsocket.c:347: warning: format '%d' expects type 'int', but argument 8 has type 'gid_t' [-Wformat] rpc/virnetsocket.c: In function 'virNetSocketGetLocalIdentity': rpc/virnetsocket.c:743: warning: pointer targets in passing argument 5 of 'getsockopt' differ in signedness * src/Makefile.am (libvirt_driver_remote_la_CFLAGS) (libvirt_net_rpc_client_la_CFLAGS) (libvirt_net_rpc_server_la_CFLAGS): Include XDR_CFLAGS, for rpc headers on cygwin. * src/rpc/virnetsocket.c (virNetSocketNewListenUNIX) (virNetSocketGetLocalIdentity): Avoid compiler warnings. 2011-07-29 Eric Blake build: avoid non-portable shell in test setup POSIX states that 'a=1; a=2 b=$a command' has unspecified results for the value of $b visible within command. In particular, on BSD, this resulted in PATH not picking up the in-test ssh. * tests/Makefile.am (lv_abs_top_builddir): New macro. (path_add, TESTS_ENVIRONMENT): Use it to avoid referring to an environment variable set previously within the same command line. Reported by Matthias Bolte. 2011-07-29 Matthias Bolte tests: Don't use bash if we don't have to This tested failed on FreeBSD because it was using bash, that might not be installed. 2011-07-29 Osier Yang utils: More useful error message for hook script failure Commit 3709a386 ported hooks codes to new command execution API, together with the useful error message removed. Though we can't get "errbuf" from the new command execution API anymore, still we can give a more useful error. https://bugzilla.redhat.com/show_bug.cgi?id=726398 2011-07-29 Matthias Bolte freebsd: Fix build problem due to picking up the wrong libvirt.h Gettext annoyingly modifies CPPFLAGS in-place, putting -I/usr/local/include into the search patch if libintl headers must be used from that location. But since we must support automake 1.9.6 which lacks AM_CPPFLAGS, and since CPPFLAGS is used prior to INCLUDES, this means that the build picks up the _old_ installed libvirt.h in priority to the in-tree version, leading to all sorts of weird build failures on FreeBSD. Fix this by teaching configure to undo gettext's actions, but to keep any changes required by gettext at the end of INCLUDES after all in-tree locations are used first. Also requires adding a wrapper Makefile.am and making gnulib-tool create just gnulib.mk files during the bootstrap process. 2011-07-29 Matthias Bolte freebsd: Avoid /bin/true in commandtest Rely on PATH and use just true, because on FreeBSD it's /usr/bin/true. 2011-07-29 Matthias Bolte tests: Unify style of test skipping code Prefer 'return EXIT_AM_SKIP' over 'exit(EXIT_AM_SKIP)'. Prefer 'int main(void)' over 'int main(int argc, char **argv)'. Fix mymain signature in commandtest and nodeinfotest. 2011-07-28 Eric Blake save: let qemu driver manipulate save files The goal here is that save-image-dumpxml fed back to save-image-define should not change the save file; anywhere that this is not the case is probably a bug in domain_conf.c. * src/qemu/qemu_driver.c (qemuDomainSaveImageGetXMLDesc) (qemuDomainSaveImageDefineXML): New functions. (qemuDomainSaveImageOpen): Add parameter. (qemuDomainRestoreFlags, qemuDomainObjRestore): Adjust clients. 2011-07-28 Eric Blake save: support qemu modifying xml on domain save/restore With this, it is possible to update the path to a disk backing image on either the save or restore action, without having to binary edit the XML embedded in the state file. This also modifies virDomainSave to output a smaller xml (only the inactive xml, which is all the more virDomainRestore parses), while still guaranteeing padding for most typical abi-compatible xml replacements, necessary so that the next patch for virDomainSaveImageDefineXML will not cause unnecessary modifications to the save image file. * src/qemu/qemu_driver.c (qemuDomainSaveInternal): Add parameter, only use inactive state, and guarantee padding. (qemuDomainSaveImageOpen): Add parameter. (qemuDomainSaveFlags, qemuDomainManagedSave) (qemuDomainRestoreFlags, qemuDomainObjRestore): Update callers. 2011-07-28 Eric Blake maint: add missing copyright notices I went with the shorter license notice used by src/libvirt.c, rather than spelling out the full LGPLv2+ clause into each of these files. * configure.ac: Declare copyright. * all Makefile.am: Likewise. 2011-07-28 Eric Blake xen: drop unused callbacks Found by: for f in $(sed -n 's/.*Drv[^ ]* \([^;]*\);.*/\1/p' src/xen/xen_driver.h) do git grep "\(\.\|->\)$f\b" src/xen done | cat and looking through the resulting list to see which callback struct members are still necessary. * src/xen/xen_driver.h (xenUnifiedDriver): Drop all callbacks that are only used directly. * src/xen/xen_hypervisor.c (xenHypervisorDriver): Shrink list. * src/xen/xen_inotify.c (xenInotifyDriver): Likewise. * src/xen/xend_internal.c (xenDaemonDriver): Likewise. * src/xen/xm_internal.c (xenXMDriver): Likewise. * src/xen/xs_internal.c (xenStoreDriver): Likewise. 2011-07-28 Eric Blake xen: make direct call when there is only one subdriver No need to use a for loop if we know there is exactly one client. Found by: for f in $(sed -n 's/.*Drv[^ ]* \([^;]*\);.*/\1/p' src/xen/xen_driver.h) do git grep "\(\.\|->\)$f\b" src/xen done | cat and looking through the resulting list to see which callback struct members are used exactly once. The next patch will ensure that we don't reintroduce uses of these callbacks. * src/xen/xen_driver.c (xenUnifiedClose): Call close unconditionally, to match xenUnifiedOpen. (xenUnifiedNodeGetInfo, xenUnifiedDomainCreateXML) (xenUnifiedDomainSave, xenUnifiedDomainRestore) (xenUnifiedDomainCoreDump, xenUnifiedDomainUpdateDeviceFlags): Make direct call to lone implementation. * src/xen/xend_internal.h (xenDaemonDomainCoreDump) (xenDaemonUpdateDeviceFlags, xenDaemonCreateXML): Add prototypes. * src/xen/xend_internal.c (xenDaemonDomainCoreDump) (xenDaemonUpdateDeviceFlags, xenDaemonCreateXML): Export. 2011-07-28 Eric Blake xen: reduce callback special cases The callback struct is great when iterating through several possibilities, but when calling a known callback, it's just overhead. We can make the direct call in those cases. * src/xen/xen_driver.c (xenUnifiedOpen, xenUnifiedDomainSuspend) (xenUnifiedDomainResume, xenUnifiedDomainDestroyFlags): Make direct calls instead of going through callback. 2011-07-28 Eric Blake xen: cleanup callback struct Using C99 initializers and xen-specific prefixes will make it so that future patches are less likely to add callback members to the xenUnifiedDriver struct, since the goal is to get rid of the callback struct in the first place. * src/xen/xen_driver.h (xenUnifiedDriver): Rename all struct members, to make it obvious which ones are still in use. * src/xen/xen_driver.c: Update all callers. * src/xen/xen_hypervisor.c (xenHypervisorDriver): Rewrite with C99 initializers. * src/xen/xend_internal.c (xenDaemonDriver): Likewise. * src/xen/xs_internal.c (xenStoreDriver): Likewise. * src/xen/xm_internal.c (xenXMDriver): Likewise. * src/xen/xen_inotify.c (xenInotifyDriver): Likewise. 2011-07-28 Laine Stump libxl: fix build failure due to change in virDomainGraphicsDef This failure was introduced by commit dacee3d, which removed listenAddr from the unions in virDomainGraphicsDef in favor of putting it in the address attribute of virDomainGraphicsListenDef. 2011-07-28 Laine Stump qemu: support type=network in domain graphics The domain XML now understands the subelement of its element (including when listen type='network'), and the network driver has an internal API that will turn a network name into an IP address, so the final logical step is to put the glue into the qemu driver so that when it is starting up a domain, if it finds in the XML, it will call the network driver to get an IPv4 address associated with network xyz, and tell qemu to listen for vnc (or spice) on that address rather than the default address (localhost). The motivation for this is that a large installation may want the guests' VNC servers listening on physical interfaces rather than localhost, so that users can connect directly from the outside; this requires sending qemu the appropriate IP address to listen on. But this address will of course be different for each host, and if a guest might be migrated around from one host to another, it's important that the guest's config not have any information embedded in it that is specific to one particular host. , or host bridge network: 2) in the element of each guest's domain xml, tell vnc to listen on the network name used in step 1: (all the above also applies for graphics type='spice'). 2011-07-28 Laine Stump conf: add subelement to domain element Once it's plugged in, the element will be an optional replacement for the "listen" attribute that graphics elements already have. If the element is type='address', it will have an attribute called 'address' which will contain an IP address or dns name that the guest's display server should listen on. If, however, type='network', the element should have an attribute called 'network' that will be set to the name of a network configuration to get the IP address from. * docs/schemas/domain.rng: updated to allow the element * docs/formatdomain.html.in: document the element and its attributes. * src/conf/domain_conf.[hc]: 1) The domain parser, formatter, and data structure are modified to support 0 or more subelements to each element. The old style "legacy" listen attribute is also still accepted, and will be stored internally just as if it were a separate element. On output (i.e. format), the address attribute of the first element of type 'address' will be duplicated in the legacy "listen" attribute of the element. 2) The "listenAddr" attribute has been removed from the unions in virDomainGRaphicsDef for graphics types vnc, rdp, and spice. This attribute is now in the subelement (aka virDomainGraphicsListenDef) 3) Helper functions were written to provide simple access (both Get and Set) to the listen elements and their attributes. * src/libvirt_private.syms: export the listen helper functions * src/qemu/qemu_command.c, src/qemu/qemu_hotplug.c, src/qemu/qemu_migration.c, src/vbox/vbox_tmpl.c, src/vmx/vmx.c, src/xenxs/xen_sxpr.c, src/xenxs/xen_xm.c Modify all these files to use the listen helper functions rather than directly referencing the (now missing) listenAddr attribute. There can be multiple elements to a single , but the drivers all currently only support one, so all replacements of direct access with a helper function indicate index "0". * tests/* - only 3 of these are new files added explicitly to test the new element. All the others have been modified to reflect the fact that any legacy "listen" attributes passed in to the domain parse will be saved in a element (i.e. one of the virDomainGraphicsListenDefs), and during the domain format function, both the element as well as the legacy attributes will be output. 2011-07-28 Alex Jia virsh: avoid missing zero value judgement in cmdBlkiotune * tools/virsh.c: fix missing zero value judgement in cmdBlkiotune and correct vshError information. when weight is equal to 0, the cmdBlkiotune will not raise any error information when judge weight value first time, and execute else branch to judge weight value again, strncpy(temp->field, VIR_DOMAIN_BLKIO_WEIGHT, sizeof(temp->field)) will be not executed for ever. However, if and only if param->field is equal to VIR_DOMAIN_BLKIO_WEIGHT, underlying qemuDomainSetBlkioParameters function will check whether weight value is in range [100, 1000]. * how to reproduce? % virsh blkiotune ${guestname} --weight 0 2011-07-28 Eric Blake build: avoid type-punning compiler warning On RHEL 5, with gcc 4.1.2: rpc/virnetsaslcontext.c: In function 'virNetSASLSessionUpdateBufSize': rpc/virnetsaslcontext.c:396: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] * src/rpc/virnetsaslcontext.c (virNetSASLSessionUpdateBufSize): Use a union to work around gcc warning. 2011-07-28 Eric Blake qemu: fix nested job with driver lock held qemuMigrationUpdateJobStatus (called in a loop by migration and save tasks) uses qemuDomainObjEnterMonitorWithDriver; however, that function ended up starting a nested job without releasing the driver. Since no one else is making nested calls, we can inline the internal functions to properly track driver_locked. * src/qemu/qemu_domain.h (qemuDomainObjBeginNestedJob) (qemuDomainObjBeginNestedJobWithDriver) (qemuDomainObjEndNestedJob): Drop unused prototypes. * src/qemu/qemu_domain.c (qemuDomainObjEnterMonitorInternal): Reflect driver lock to nested job. (qemuDomainObjBeginNestedJob) (qemuDomainObjBeginNestedJobWithDriver) (qemuDomainObjEndNestedJob): Drop unused functions. 2011-07-28 Michal Privoznik qemu: Fix memory leak on metadata fetching As written in virStorageFileGetMetadataFromFD decription, caller must free metadata after use. Qemu driver miss this and therefore leak metadata which can grow to huge mem leak if somebody query for blockInfo a lot. 2011-07-28 Matthias Bolte freebsd: Add gnulib environ module for the commandtest 2011-07-28 Michal Privoznik libvirt.c: Update outdated description of flags Because we do support flags for virDomainSetBlkioParameters and virDomainGetBlkioParameters, update appropriate description as well. 2011-07-28 Hu Tao python: add python binding for virDomainSetMemoryParameters python: add python binding for virDomainGetMemoryParameters python: add python binding for virDomainSetBlkioParameters python: add python binding for virDomainGetBlkioParameters 2011-07-28 Alex Jia virsh: fix memory leak in cmdVolPath code * tools/virsh.c: avoid memory leak in cmdVolPath. * src/libvirt.c: Add doc for virStorageVolGetPath to tell one must free() the returned path after use. * how to reproduce? % dd if=/dev/zero of=/var/lib/libvirt/images/foo.img count=1 bs=10M % virsh pool-refresh default % valgrind -v --leak-check=full virsh vol-path --vol \ /var/lib/libvirt/images/foo.img * actual results: Detected in valgrind run: ==16436== 32 bytes in 1 blocks are definitely lost in loss record 7 of 22 ==16436== at 0x4A05FDE: malloc (vg_replace_malloc.c:236) ==16436== by 0x386A314B3D: xdr_string (in /lib64/libc-2.12.so) ==16436== by 0x3DF8CD770D: xdr_remote_nonnull_string (remote_protocol.c:3 ==16436== by 0x3DF8CD7EC8: xdr_remote_storage_vol_get_path_ret % virsh pool-refresh default % valgrind -v --leak-check=full virsh vol-path --vol \ /var/lib/libvirt/images/foo.img 2011-07-28 Osier Yang qemu: Improve docs for virsh dump format The error in getCompressionType will never be reported, change the errors codes into warning (VIR_WARN("%s", _(foo)); doesn't break syntax-check rule), and also improve the docs in qemu.conf to tell user the truth. 2011-07-28 Eric Blake qemu: improve thread documentation * src/qemu/THREADS.txt: Fix problems with typos, grammar, and outdated examples. 2011-07-27 Eric Blake virsh: expose change-protection during migration * tools/virsh.c (doMigrate): Add --change-protection flag. * tools/virsh.pod (migrate): Document it. 2011-07-27 Matthias Bolte rpc: Fix memory leak in remoteDomainSet*Parameters functions Add a new helper remoteFreeTypedParameters and teach the generator to add it to the cleanup section. https://bugzilla.redhat.com/show_bug.cgi?id=725322 2011-07-27 Lai Jiangshan send-key: Implement Python API 2011-07-27 Jiri Denemark qemu: Remove special case for virDomainAbortJob This doesn't abort migration job in any phase, yet. qemu: Remove special case for virDomainSuspend qemu: Remove special case for virDomainMigrateSetMaxDowntime Call qemu monitor command directly within a special job that is only allowed during outgoing migration. qemu: Remove special case for virDomainMigrateSetMaxSpeed Call qemu monitor command directly within a special job that is only allowed during outgoing migration. qemu: Remove special case for virDomainBlockStats Like other query commands, this can now be called directly during migration. qemu: Remove special case for virDomainGetBlockInfo Like other query commands, this can now be called directly during migration. qemu: Recover from interrupted migration 2011-07-27 Jiri Denemark qemu: Migration job on source daemon Make MIGRATION_OUT use the new helper methods. This also introduces new protection to migration v3 process: the migration job is held from Begin to Confirm to avoid changes to a domain during migration (esp. between Begin and Perform phases). This change is automatically applied to p2p and tunneled migrations. For normal migration, this requires support from a client. In other words, if an old (pre 0.9.4) client starts normal migration of a domain, the domain will not be protected against changes between Begin and Perform steps. 2011-07-27 Jiri Denemark qemu: Migration job on destination daemon Make MIGRATION_IN use the new helper methods. qemu: Implement migration job phases This patch introduces several helper methods to deal with jobs and phases during migration in a simpler manner. 2011-07-27 Eric Blake build: support warnings on RHEL 5 Without this, a configure built by autoconf 2.59 was broken when trying to detect which compiler warning flags were supported. * .gnulib: Update to latest, for warnings.m4 fix. * bootstrap.conf: Add fclose explicitly, to match recent gnulib implicit dependency changes. * src/qemu/qemu_conf.c (includes): Drop unused include. * src/uml/uml_conf.c (include): Likewise. Reported by Daniel P. Berrange. 2011-07-27 Michal Privoznik bandwidth: Integrate bandwidth into portgroups Every DomainNetDef has a bandwidth, as does every portgroup. Whenever a DomainNetDef of type NETWORK is about to be used, a call is made to networkAllocateActualDevice(). This function chooses the "best" bandwidth object and places it in the DomainActualNetDef. From that point on, whenever some code needs to use the bandwidth data for the interface, it's retrieved with virDomainNetGetActualBandwidth(), which will always return the "best" info as determined in the previous step. 2011-07-27 Osier Yang doc: Add doc for blockpull and blockjob commands Commit b31abc6f0 introduce commands blockpull and blockjob, but forgot to add the docs meanwhile. 2011-07-27 Cole Robinson python: Properly report errors if virStreamRecv fails We only want to raise the special value -2. -1 should return None which tells the bindings to throw an exception. 2011-07-26 Alex Jia tools: format percent strings of nodecpustats * tools/virsh.c: format strings display for virsh nodecpustats --percent. * how to reproduce? % virsh nodecpustats --percent usage: 2.0% user : 1.0% system: 1.0% idle : 98.0% iowait: 0.0% * after format strings % virsh nodecpustats --percent usage: 2.0% user: 1.0% system: 1.0% idle: 98.0% iowait: 0.0% 2011-07-26 Stefan Berger Build: fix build if HAVE_AVAHI is not defined Fix the build if HAVE_AVAHI is not defined. 2011-07-26 Dave Allan Fix incorrect implication about list options The description of the list command seemed to suggest that it could take a set of domains as an argument, which is not correct in the current HEAD. If virsh list is intended to take a list of domains, then this patch should be NAK'd and a bug opened against virsh list. Reported by hachi on #virt v2: Change language to include transient domains Osier pointed out that transient domains are not defined, so what I had originally proposed wasn't quite correct. 2011-07-26 Daniel P. Berrange Fix race in ref counting when handling RPC jobs When an incoming RPC message is ready for processing, virNetServerClientDispatchRead() will invoke the 'dispatchFunc' callback. This is set to virNetServerDispatchNewMessage This function puts the message + client in a queue for processing by the thread pool. The thread pool worker function is virNetServerHandleJob The first thing this does is acquire an extra reference on the 'client'. Unfortunately, between the time the message+client are put on the thread pool queue, and the time the worker runs, the client object may have had its last reference removed. We clearly need to add the reference to the client object before putting the client on the processing queue * src/rpc/virnetserverclient.c: Add a reference to the client when invoking the dispatch function * src/rpc/virnetserver.c: Don't acquire a reference to the client when in the worker thread 2011-07-26 Wen Congyang set cpu bandwidth for the vm The cpu bandwidth is applied at the vcpu group level. We should apply it at the vm group level too, because the vm may do heavy I/O, and it will affect the other vm. We apply cpu bandwidth at the vcpu and the vm group level, so we must ensure that max(child_quota) <= parent_quota when we modify cpu bandwidth. 2011-07-26 Daniel P. Berrange Fix build with gnutls 1.0.x branch Fix typos in virsh.pod file * tools/virsh.pod: Fix missing > tag in docs 2011-07-26 Osier Yang network: Fix typo Introduced by commit 239322cb, reported by Ruben Kerkhof. 2011-07-26 Daniel P. Berrange Add libtasn1-devel as a BuildRequires for libvirt.spec * libvirt.spec.in: Add libtasn1-devel 2011-07-26 Daniel P. Berrange Add mutex protection to SASL and TLS modules The virNetSASLContext, virNetSASLSession, virNetTLSContext and virNetTLSSession classes previously relied in their owners (virNetClient / virNetServer / virNetServerClient) to provide locking protection for concurrent usage. When virNetSocket gained its own locking code, this invalidated the implicit safety the SASL/TLS modules relied on. Thus we need to give them all explicit locking of their own via new mutexes. * src/rpc/virnetsaslcontext.c, src/rpc/virnettlscontext.c: Add a mutex per object 2011-07-26 Daniel P. Berrange Report error if unable to bind to any socket When setting up a server socket, we must skip EADDRINUSE errors from bind, since the IPv6 socket bind may have already bound to the IPv4 socket too. If we don't manage to bind to any sockets at all though, we should then report the EADDRINUSE error as normal. This fixes the case where libvirtd would not exit if some other program was listening on its TCP/TLS ports. * src/rpc/virnetsocket.c: Report EADDRINUSE 2011-07-26 Daniel P. Berrange Fix leak of mDNS object in virNetServer * src/rpc/virnetserver.c: Free mDNS object Fix memory leaks in MDNS code * src/rpc/virnetservermdns.c: Fix leaks 2011-07-25 Eric Blake virsh: use faster bit search Now that gnulib gives us ffs, we might as well use it. * tools/virsh.c (vshCmddefGetData): Use ffs rather than count_one_bits. 2011-07-25 Eric Blake virsh: make vcpucount use --current consistently Rename the existing --current flag to the new name --active, while adding a new flag --current to expose the new VIR_DOMAIN_AFFECT_CURRENT flag of virDomainGetVcpusFlags. For backwards compability, the output does not change (even though the label "current" no longer matches the spelling of the option that would trigger that number in isolation), and we accept "--current --live" as an undocumented synonym for "--active --live" to avoid breaking any existing clients. * tools/virsh.c (cmdVcpucount): Add --active flag, and rearrange existing flag handling to expose VIR_DOMAIN_AFFECT_CURRENT support. * tools/virsh.pod (vcpucount): Document this. 2011-07-25 Eric Blake vcpu: teach getVcpusFlags about current Now that virDomainSetVcpusFlags knows about VIR_DOMAIN_AFFECT_CURRENT, so should virDomainGetVcpusFlags. Unfortunately, the virsh counterpart 'virsh vcpucount' has already commandeered --current for a different meaning, so teaching virsh to expose this in the next patch will require a bit of care. * src/libvirt.c (virDomainGetVcpusFlags): Allow VIR_DOMAIN_AFFECT_CURRENT. * src/libxl/libxl_driver.c (libxlDomainGetVcpusFlags): Likewise. * src/qemu/qemu_driver.c (qemudDomainGetVcpusFlags): Likewise. * src/test/test_driver.c (testDomainGetVcpusFlags): Likewise. * src/xen/xen_driver.c (xenUnifiedDomainGetVcpusFlags): Likewise. 2011-07-25 Cole Robinson python: Handle embedded NUL in stream.send data Otherwise things like volume upload are only useful with text data. 2011-07-25 Laine Stump util: change virFile*Pid functions to return < 0 on failure Although most functions in libvirt return 0 on success and < 0 on failure, there are a few functions lingering around that return errno (a positive value) on failure, and sometimes code calling those functions incorrectly assumes the <0 standard. I noticed one of these the other day when auditing networkStartDhcpDaemon after Guido Gunther found a place where success was improperly returned on failure (that patch has been acked and is pending a push). The problem was that it expected the return value from virFileReadPid to be < 0 on failure, but it was actually positive (it was also neglected to set the return code in this case, similar to the bug found by Guido). This all led to the fact that *all* of the virFile*Pid functions in util.c are returning errno on failure. This patch remedies that problem by changing them all to return -errno on failure, and makes any necessary changes to callers of the functions. (In the meantime, I also properly set the return code on failure of virFileReadPid in networkStartDhcpDaemon). 2011-07-25 Guido Günther Catch dnsmasq start failures While we checked the return value we didn't maks sure ret != 0 which resulted in dnsmasq errors being ignored. 2011-07-25 Daniel P. Berrange Fix import of private key with older gnutls With older GNUTLS the gnutls_x509_privkey_import function is unable to import our private key. Instead we must use the alternative gnutls_x509_privkey_import_pkcs8() (as certtool does). * virnettlscontexttest.c: Fix import of private key with older gnutls. Also add missing newlines to key 2011-07-25 Daniel P. Berrange Fix TLS context tests with expired certs commit 5283ea9b1d8a4b0f2fd6796bf60615aca7b6c3e6 changed the semantics of the 'expire_offset' field in the test case struct so that instead of being an absolute timestamp, it was a delta relative to the current time. This broke the test cases which were testing expiry of certificates, by putting the expiry time into the future, instead of in the past. Fix this by changing the expiry values to be negative, so that the delta goes into the past again. * virnettlscontexttest.c: Fix expiry tests 2011-07-25 Wen Congyang rename cfs_* to vcpu_* In the XML file we now have 1024 90000 0 But the schedinfo parameter are being named cpu_shares: 1024 cfs_period: 90000 cfs_quota: 0 The period/quota is per-vcpu value, so these new tunables should be named 'vcpu_period' and 'vcpu_quota'. 2011-07-25 Cole Robinson docs: Break up 'Basic Resources' XML section We had a bit too many elements crammed in there. Separate it into different headings: - CPU Allocation () - CPU Tuning () - Memory allocation ( and ) - Memory backing () - Memory tuning () - Numa tuning () - Block I/O tuning () 2011-07-25 Taku Izumi python: add Python binding for virDomainGetVcpuPinInfo API This patch adds the Python bindings for virDomainGetVcpuPinInfo API. * python/generator.py: add it to generator skip list * python/libvirt-override-api.xml: provide an override description * python/libvirt-override.c: provide an override binding implementation python: add Python binding for virDomainPinVcpusFlags API This patch adds the Python bindings for virDomainPinVcpuFlags API. * python/generator.py: add it to the generator skip list * python/libvirt-override-api.xml: provide override description * python/libvirt-override.c: provide override bindings implementation python: add Python binding for virDomainGetSchedulerParametersFlags API This patch adds the Python bindings for virDomainGetSchedulerParametersFlags API. python: add Python binding for virDomainGetSchedulerParametersFlags API This patch adds the Python bindings for virDomainGetSchedulerParametersFlags API. * python/libvirt-override-api.xml: provide and override description * python/libvirt-override.c: implement the bindings 2011-07-25 Eric Blake tests: detect gnutls errors * tests/virnettlscontexttest.c (testTLSLoadKey): Report errors. 2011-07-25 Wieland Hoffmann driver.h: Fix two driver documentation mistakes 2011-07-25 Michal Privoznik bandwidth: Add domain schema and xml2xml tests bandwidth: Add test cases for network bandwidth: Implement functions to enable and disable QoS These function executes 'tc' with appropriate arguments to set desired QoS setting on interface or bridge during its creation. bandwidth: Create format functions 2011-07-25 Michal Privoznik bandwidth: Add parsing and free functions These functions parse given XML node and return pointer to the output. Unknown elements are silently ignored. Attributes must be integer and must fit in unsigned long long. Free function frees elements of virBandwidth structure. 2011-07-25 Michal Privoznik bandwidth: Declare internal structures 2011-07-25 Michal Privoznik bandwidth: Define schema and create documentation Define new 'bandwidth' element with possible child element 'inbound' and 'outbound' addressing incoming and outgoing traffic respectively: Leaving any element out means not to shape traffic in that direction. The units for average and peak (rate) are in kilobytes per second, for burst (size) are just in kilobytes. This element can be inserted into domain's 'interface' and 'network'. 2011-07-25 Laine Stump network: provide internal API to return IP of a network The new listenNetwork attribute needs to learn an IP address based on a named network. This patch provides a function networkGetNetworkAddress which provides that. Some networks have an IP address explicitly in their configuration (ie, those with a forward type of "none", "route", or "nat"). For those, we can just return the IP address from the config. The rest will have a physical device associated with them (either via , , or possibly via a pool of interfaces inside the network's element) and we will need to ask the kernel for a current IP address of that device (via the newly added ifaceGetIPAddress) If networkGetNetworkAddress encounters an error while trying to learn the address for a network, it will return -1. In the case that libvirt has been compiled without the network driver, the call is a macro which reduces to -2. This allows differentiating between a failure of the network driver, and its complete absence. 2011-07-25 Laine Stump util: add an ifaceGetIPAddress to the interface utilities This function uses ioctl(SIOCGIFADDR), which limits it to returning the first IPv4 address of an interface, but that's what we want right now (the place we're going to use the address only accepts one). 2011-07-25 Eric Blake tests: fix compilation failures Even though gnutls is a hard-req for libvirt, and gnutls depends on libtasn1, that does not mean that you have to have the libtasn1 development files installed. Skip the test rather than failing compilation in that case. With newer gcc, the test consumed too much stack space. Move things to static storage to fix that. * configure.ac (AC_CHECK_HEADERS): Check for libtasn1.h. (HAVE_LIBTASN1): New automake conditional. * tests/Makefile.am (virnettlsconvirnettlscontexttest_SOURCES) (virnettlscontexttest_LDADD): Allow compilation without libtasn1. * tests/virnettlscontexttest.c: Skip test if headers not present. (struct testTLSCertReq): Alter time members. (testTLSGenerateCert): Reflect the change. (mymain): Reduce stack usage. 2011-07-22 Daniel P. Berrange Pre-create /var/lib/libvirt/sanlock directory The sanlock plugin for libvirt expects the directory /var/lib/libvirt/sanlock to exist. Create this and add it to the RPM * libvirt.spec.in: Add /var/lib/libvirt/sanlock * src/Makefile.am: Create /var/lib/libvirt/sanlock 2011-07-22 Daniel P. Berrange Honour filesystem readonly flag & make special FS readonly A container should not be allowed to modify stuff in /sys or /proc/sys so make them readonly. Make /selinux readonly so that containers think that selinux is disabled. Honour the readonly flag when mounting container filesystems from the guest XML config * src/lxc/lxc_container.c: Support readonly mounts 2011-07-22 Daniel P. Berrange Refactor mounting of special filesystems Even in non-virtual root filesystem mode we should be mounting more than just a new /proc. Refactor lxcContainerMountBasicFS so that it does everything except for /dev and /dev/pts moving that into lxcContainerMountDevFS. Pass in a source prefix to lxcContainerMountBasicFS() so it can be used in both shared root and private root modes. * src/lxc/lxc_container.c: Unify mounting code for special filesystems 2011-07-22 Daniel P. Berrange Pull code for doing a bind mount into separate method The bind mount setup is about to get more complicated. To avoid having to deal with several copies, pull it out into a separate lxcContainerMountFSBind method. Also pull out the iteration over container filesystems, so that it will be easier to drop in support for non-bind mount filesystems * src/lxc/lxc_container.c: Pull bind mount code out into lxcContainerMountFSBind 2011-07-22 Daniel P. Berrange Fix typos in daemon config file from previous commit 2011-07-22 Daniel P. Berrange Add a test case for certificate validation This test case checks certification validation rules for - Basic constraints - Key purpose - Key usage - Start/expiry times It checks initial context creation sanity checks, and live session validation 2011-07-22 Daniel P. Berrange Allow certificate sanity checking to be disabled When libvirtd starts it it will sanity check its own certs, and before libvirt clients connect to a remote server they will sanity check their own certs. This patch allows such sanity checking to be skipped. There is no strong reason to need to do this, other than to bypass possible libvirt bugs in sanity checking, or for testing purposes. libvirt.conf gains tls_no_sanity_certificate parameter to go along with tls_no_verify_certificate. The remote driver client URIs gain a no_sanity URI parameter * daemon/test_libvirtd.aug, daemon/libvirtd.conf, daemon/libvirtd.c, daemon/libvirtd.aug: Add parameter to allow cert sanity checks to be skipped * src/remote/remote_driver.c: Add no_sanity parameter to skip cert checks * src/rpc/virnettlscontext.c, src/rpc/virnettlscontext.h: Add new parameter for skipping sanity checks independantly of skipping session cert validation checks 2011-07-22 Matthias Bolte build: Use $(PYTHON) instead of python for the keycode map generator Also prepend $(AM_V_GEN) to the command line, mark virkeycode-mapgen.py as executable and switch the shebang line from /bin/python to the commonly use /usr/bin/python. 2011-07-22 Matthias Bolte xenapi: Fix double-freeing the session in xenapiClose xen_session_logout already frees the whole session object. Don't call xenSessionFree on a freed session object. Reported by Sharmila Radhakrishnan. 2011-07-22 Oskari Saarenmaa remote/ssh: optional "keyfile" parameter. New optional parameter "keyfile" for ssh transport allows the user to select the private key to be used to authenticate to the remote host. 2011-07-22 Laine Stump util: make interface.c functions consistently return < 0 on error All of the functions in util/interface.c were returning 0 on success, but some returned -1 on error, and some returned a positive value (usually the value of errno, but sometimes just 1). Libvirt's standard is to return < 0 on error (in the case of functions that need to return errno, -errno is returned. This patch modifies all functions in interface.c to consistently return < 0 on error, and makes changes to callers of those functions where necessary. 2011-07-22 Daniel P. Berrange Refactor the certification validation code There is some commonality between the code for sanity checking certs when initializing libvirt and the code for validating certs during a live TLS session handshake. This patchset splits up the sanity checking function into several smaller functions each doing a specific type of check. The cert validation code is then updated to also call into these functions * src/rpc/virnettlscontext.c: Refactor cert validation code 2011-07-22 Daniel P. Berrange Remove call to deprecated gnutls_certificate_type_set_priority (again) The gnutls_certificate_type_set_priority method is deprecated. Since we already set the default gnutls priority, it was not serving any useful purpose and can be removed * src/rpc/virnettlscontext.c: Remove gnutls_certificate_type_set_priority call 2011-07-22 Daniel P. Berrange Ensure that libvirtd shuts down if initialization fails If the virStateInitialize call fails we must shutdown libvirtd since drivers will not be available. Just free'ing the virNetServer is not sufficient, we must send a SIGTERM to ourselves so that we interrupt the event loop and trigger a orderly shutdown * daemon/libvirtd.c: Kill ourselves if state init fails * src/rpc/virnetserver.c: Add some debugging to event loop 2011-07-22 Adam Litke Asynchronous event for BlockJob completion When an operation started by virDomainBlockPull completes (either with success or with failure), raise an event to indicate the final status. This API allow users to avoid polling on virDomainGetBlockJobInfo if they would prefer to use an event mechanism. * daemon/remote.c: Dispatch events to client * include/libvirt/libvirt.h.in: Define event ID and callback signature * src/conf/domain_event.c, src/conf/domain_event.h, src/libvirt_private.syms: Extend API to handle the new event * src/qemu/qemu_driver.c: Connect to the QEMU monitor event for block_stream completion and emit a libvirt block pull event * src/remote/remote_driver.c: Receive and dispatch events to application * src/remote/remote_protocol.x: Wire protocol definition for the event * src/remote_protocol-structs: structure definitions for protocol verification * src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h, src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event from QEMU monitor 2011-07-22 Adam Litke Enable virDomainBlockPull in the python API virDomainGetBlockJobInfo requires manual override since it returns a custom type. * python/generator.py: reenable bindings for this entry point * python/libvirt-override-api.xml python/libvirt-override.c: manual overrides 2011-07-22 Adam Litke Enable the virDomainBlockPull API in virsh Define two new virsh commands: * blockpull: Initiate a blockPull for the given disk * blockjob: Retrieve progress info, modify speed, and cancel active block jobs Share print_job_progress() with the migration code. * tools/virsh.c: implement the new commands 2011-07-22 Adam Litke Implement virDomainBlockPull for the qemu driver The virDomainBlockPull* family of commands are enabled by the following HMP/QMP commands: 'block_stream', 'block_job_cancel', 'info block-jobs' / 'query-block-jobs', and 'block_job_set_speed'. * src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk streaming by using the proper qemu monitor commands. * src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor 2011-07-22 Adam Litke Add virDomainBlockPull support to the remote driver The generator can handle everything except virDomainGetBlockJobInfo(). * src/remote/remote_protocol.x: provide defines for the new entry points * src/remote/remote_driver.c daemon/remote.c: implement the client and server side for virDomainGetBlockJobInfo. * src/remote_protocol-structs: structure definitions for protocol verification * src/rpc/gendispatch.pl: Permit some unsigned long parameters 2011-07-22 Adam Litke virDomainBlockPull: Implement the main entry points * src/libvirt.c: implement the main entry points 2011-07-22 Adam Litke Add new API virDomainBlockPull* to headers Set up the types for the block pull functions and insert them into the virDriver structure definition. Symbols are exported in this patch to prevent documentation compile failures. * include/libvirt/libvirt.h.in: new API * src/driver.h: add the new entry to the driver structure * python/generator.py: fix compiler errors, the actual python bindings * are implemented later * src/libvirt_public.syms: export symbols * docs/apibuild.py: Extend 'unsigned long' parameter exception to this * API 2011-07-22 Eric Blake save: add virsh commands for manipulating save files Now you can edit a saved state file even if you forgot to grab a dumpxml file prior to saving a domain. Plus, in-place editing feels so much nicer. * tools/virsh.c (cmdSaveImageDumpxml, cmdSaveImageDefine) (cmdSaveImageEdit): New commands. * tools/virsh.pod (save-image-dumpxml, save-image-define) (save-image-edit): Document them. 2011-07-22 Eric Blake save: wire up remote protocol * src/remote/remote_driver.c (remote_driver): Add new callbacks. * src/remote/remote_protocol.x (remote_procedure): New RPCs. (remote_domain_save_image_get_xml_desc_args) (remote_domain_save_image_get_xml_desc_ret) (remote_domain_save_image_define_xml_args): New structs. * src/remote_protocol-structs: Update. 2011-07-22 Eric Blake save: new API to manipulate save file images Modifying the xml on either save or restore only gets you so far - you have to remember to 'virsh dumpxml dom' just prior to the 'virsh save' in order to have an xml file worth modifying that won't be rejected due to abi breaks. To make this more powerful, we need a way to grab the xml embedded within a state file, and from there, it's not much harder to also support modifying a state file in-place. Also, virDomainGetXMLDesc didn't document its flags. * include/libvirt/libvirt.h.in (virDomainSaveImageGetXMLDesc) (virDomainSaveImageDefineXML): New prototypes. * src/libvirt.c (virDomainSaveImageGetXMLDesc) (virDomainSaveImageDefineXML): New API. * src/libvirt_public.syms: Export them. * src/driver.h (virDrvDomainSaveImageGetXMLDesc) (virDrvDomainSaveImgeDefineXML): New driver callbacks. 2011-07-22 Eric Blake save: support bypass-cache flag in libvirt-guests init script libvirt-guests is a perfect use case for bypassing the file system cache - lots of filesystem traffic done at system shutdown, where caching is pointless, and startup, where reading large files only once just gets in the way. Make this a configurable option in the init script, but defaulting to existing behavior. * tools/libvirt-guests.sysconf (BYPASS_CACHE): New variable. * tools/libvirt-guests.init.sh (start, suspend_guest): Use it. 2011-07-22 Eric Blake save: support bypass-cache flag in qemu.conf When auto-dumping a domain on crash events, or autostarting a domain with managed save state, let the user configure whether to imply the bypass cache flag. * src/qemu/qemu.conf (auto_dump_bypass_cache, auto_start_bypass_cache): Document new variables. * src/qemu/libvirtd_qemu.aug (vnc_entry): Let augeas parse them. * src/qemu/qemu_conf.h (qemud_driver): Store new preferences. * src/qemu/qemu_conf.c (qemudLoadDriverConfig): Parse them. * src/qemu/qemu_driver.c (processWatchdogEvent, qemuAutostartDomain): Honor them. 2011-07-22 Eric Blake save: support BYPASS_CACHE during qemu save/restore Wire together the previous patches to support file system cache bypass during API save/restore requests in qemu. * src/qemu/qemu_driver.c (qemuDomainSaveInternal, doCoreDump) (qemudDomainObjStart, qemuDomainSaveImageOpen, qemuDomainObjRestore) (qemuDomainObjStart): Add parameter. (qemuDomainSaveFlags, qemuDomainManagedSave, qemudDomainCoreDump) (processWatchdogEvent, qemudDomainStartWithFlags, qemuAutostartDomain) (qemuDomainRestoreFlags): Update callers. 2011-07-22 Eric Blake save: add virFileDirectFd wrapper type O_DIRECT has stringent requirements. Rather than make lots of changes at each site that wants to use O_DIRECT, it is easier to offload the work through a helper process that mirrors the I/O between a pipe and the actual direct fd, so that the other end of the pipe no longer has to worry about constraints. Plus, if the kernel ever gains better posix_fadvise support, then we only have to touch a single file to let all callers benefit from a more efficient way to avoid file system caching. * src/util/virfile.h (virFileDirectFdFlag, virFileDirectFdNew) (virFileDirectFdClose, virFileDirectFdFree): New prototypes. * src/util/virdirect.c: Implement new wrapper object. * src/libvirt_private.syms (virfile.h): Export new symbols. * cfg.mk (useless_free_options): Add to list. * po/POTFILES.in: Add new translations. 2011-07-22 Eric Blake save: let iohelper work on O_DIRECT fds Required for a coming patch where iohelper will operate on O_DIRECT fds. There, the user-space memory must be aligned to file system boundaries (at least 512, but using page-aligned works better, and some file systems prefer 64k). Made tougher by the fact that VIR_ALLOC won't work on void *, but posix_memalign won't work on char * and isn't available everywhere. This patch makes some simplifying assumptions - namely, output to an O_DIRECT fd will only be attempted on an empty seekable file (hence, no need to worry about preserving existing data on a partial block, and ftruncate will work to undo the effects of having to round up the size of the last block written), and input from an O_DIRECT fd will only be attempted on a complete seekable file with the only possible short read at EOF. * configure.ac (AC_CHECK_FUNCS_ONCE): Check for posix_memalign. * src/util/iohelper.c (runIO): Use aligned memory, and handle quirks of O_DIRECT on last write. 2011-07-22 Eric Blake save: let iohelper handle inherited fd Rather than making the iohelper subject to a race in reopening the file, it is nicer to pass an already-open fd by inheritance. The old synopsis form must continue to work - if someone updates their libvirt package and installs a new libvirt_iohelper but without restarting the old libvirtd daemon, then the daemon can still make calls using the old syntax but the new iohelper. * src/util/iohelper.c (runIO): Split code for open... (prepare): ...to new function. (usage): Update synopsis. (main): Allow alternate calling form. * src/fdstream.c (virFDStreamOpenFileInternal): Use alternate form. 2011-07-22 Eric Blake save: support --xml to virsh save/restore Also, migrate was missing documentation for the --xml option added in commit ec5301cb. * tools/virsh.c (cmdSave, cmdRestore): Add xml argument. * tools/virsh.pod (save, restore, migrate): Document it. 2011-07-22 Eric Blake save: add --bypass-cache flag to virsh save/restore operations Wire up the new flag to several virsh commands. Also, the 'dump' command had undocumented flags. * tools/virsh.c (cmdSave, cmdManagedSave, cmdDump, cmdStart) (cmdRestore): Add new flag. * tools/virsh.pod (save, managedsave, dump, start, restore): Document flags. 2011-07-22 Eric Blake save: wire up trivial save/restore flags implementations For all hypervisors that support save and restore, the new API now performs the same functions as the old. VBox is excluded from this list, because its existing domainsave is broken (there is no corresponding domainrestore, and there is no control over the filename used in the save). A later patch should change vbox to use its implementation for managedsave, and teach start to use managedsave results. * src/libxl/libxl_driver.c (libxlDomainSave): Move guts... (libxlDomainSaveFlags): ...to new function. (libxlDomainRestore): Move guts... (libxlDomainRestoreFlags): ...to new function. * src/test/test_driver.c (testDomainSave, testDomainSaveFlags) (testDomainRestore, testDomainRestoreFlags): Likewise. * src/xen/xen_driver.c (xenUnifiedDomainSave) (xenUnifiedDomainSaveFlags, xenUnifiedDomainRestore) (xenUnifiedDomainRestoreFlags): Likewise. * src/qemu/qemu_driver.c (qemudDomainSave, qemudDomainRestore): Rename and move guts. (qemuDomainSave, qemuDomainSaveFlags, qemuDomainRestore) (qemuDomainRestoreFlags): ...here. (qemudDomainSaveFlag): Rename... (qemuDomainSaveInternal): ...to this, and update callers. 2011-07-22 Eric Blake error: add new error type for reflecting partial API support VIR_ERR_INVALID_ARG implies that an argument cannot possibly be correct, given the current state of the API. VIR_ERR_CONFIG_UNSUPPORTED implies that a configuration is wrong, but arguments aren't configuration. VIR_ERR_NO_SUPPORT implies that a function is completely unimplemented. But in the case of a function that is partially implemented, yet the full power of the API is not available for that driver, none of the above messages make sense. Hence a new error message, implying that the argument is known to comply with the current state of the API, and that while the driver supports aspects of the function, it does not support that particular use of the argument. A good use case for this is a driver that supports virDomainSaveFlags, but not the dxml argument of that API. It might be feasible to also use this new error for all functions that check flags, and which accept fewer flags than what is possible in the public API. But doing so would get complicated, since neither libvirt.c nor the remote driver may do flag filtering, and every other driver would have to do a two-part check, first using virCheckFlags on all public flags (which gives VIR_ERR_INVALID_ARG for an impossible flag), followed by a particular mask check for VIR_ERR_ARGUMENT_UNSUPPORTED (for a possible public flag but unsupported by this driver). * include/libvirt/virterror.h (VIR_ERR_ARGUMENT_UNSUPPORTED): New error. * src/util/virterror.c (virErrorMsg): Give it a message. Suggested by Daniel P. Berrange. 2011-07-21 Eric Blake build: fix bugs with destroyFlags patches Build failure on xenapi_driver from compiler warnings (flags was unused). Build failure on xen (incorrect number of arguments). And in fixing that, I obeyed the comments of struct xenUnifiedDriver that state that we want to minimize the number of callback functions in that struct, not add to it. * src/xen/xen_driver.c (xenUnifiedDomainDestroyFlags): Use correct arguments. (xenUnifiedDomainDestroy): Simplify. * src/xen/xen_driver.h (xenUnifiedDriver): Remove unused callback. * src/xen/xen_hypervisor.c (xenHypervisorDestroyDomain): Likewise. * src/xen/xend_internal.c (xenDaemonDomainDestroy): Likewise. * src/xen/xend_internal.h (xenDaemonDomainDestroyFlags): Likewise. * src/xen/xm_internal.c (xenXMDriver): Likewise. * src/xen/xs_internal.c (xenStoreDriver): Likewise. * src/xen/xen_inotify.c (xenInotifyDriver): Likewise. * src/xenapi/xenapi_driver.c (xenapiDomainDestroyFlags): Reject unknown flags. 2011-07-21 Michal Privoznik remote: Fix forgotten comma during conflict resolving 2011-07-21 Laine Stump network: internal API functions to manage assignment of physdev to guest The network driver needs to assign physical devices for use by modes that use macvtap, keeping track of which physical devices are in use (and how many instances, when the devices can be shared). Three calls are added: networkAllocateActualDevice - finds a physical device for use by the domain, and sets up the virDomainActualNetDef accordingly. networkNotifyActualDevice - assumes that the domain was already running, but libvirtd was restarted, and needs to be notified by each already-running domain about what interfaces they are using. networkReleaseActualDevice - decrements the usage count of the allocated physical device, and frees the virDomainActualNetDef to avoid later accidentally using the device. bridge_driver.[hc] - the new APIs. When WITH_NETWORK is false, these functions are all #defined to be "0" in the .h file (effectively becoming a NOP) to prevent link errors. qemu_(command|driver|hotplug|process).c - add calls to the above APIs in the appropriate places. tests/Makefile.am - we need to include libvirt_driver_network.la whenever libvirt_driver_qemu.la is linked, to avoid unreferenced symbols (in functions that are never called by the test programs...) 2011-07-21 Laine Stump qemu: use virDomainNetGetActual*() in qemuDomainXMLToNative This is the one function outside of domain_conf.c that plays around with (even modifying) the internals of the virDomainNetDef, and thus can't be fixed up simply by replacing direct accesses to the fields of the struct with the GetActual*() access functions. In this case, we need to check if the defined type is "network", and if it is *then* check the actual type; if the actual type is "bridge", then we can at least put the bridgename in a place where it can be used; otherwise (if type isn't "bridge"), we behave exactly as we used to - just null out *everything*. 2011-07-21 Laine Stump qemu: use virDomainNetGetActual*() functions where appropriate The qemu driver accesses fields in the virDomainNetDef directly, but with the advent of the virDomainActualNetDef, some pieces of information may be found in a different place (the ActualNetDef) if the network connection is of type='network' and that network is of forward type='bridge|private|vepa|passthrough'. The previous patch added functions to mask this difference from callers - they hide the decision making process and just pick the value from the proper place. This patch uses those functions in the qemu driver as a first step in making qemu work with the new network types. At this point, the virDomainActualNetDef is guaranteed always NULL, so the GetActualX() function will return exactly what the def->X that's being replaced would have returned (ie bisecting is not compromised). There is one place (in qemu_driver.c) where the internal details of the NetDef are directly manipulated by the code, so the GetActual functions cannot be used there without extra additional code; that file will be treated in a separate patch. 2011-07-21 Laine Stump network: separate Start/Shutdown functions for new network types Previously all networks were composed of bridge devices created and managed by libvirt, and the same operations needed to be done for all of them when they were started and stopped (create and start the bridge device, configure its MAC address and IP address, add iptables rules). The new network types are (for now at least) managed outside of libvirt, and the network object is used only to contain information about the network, which is then used as each individual guest connects itself. This means that when starting/stopping one of these new networks, we really want to do nothing, aside from marking the network as active/inactive. This has been setup as toplevel Start/Shutdown functions that do the small bit of common stuff, then have a switch statement to execute network type-specific start/shutdown code, then do a bit more common code. The type-specific functions called for the new host bridge and macvtap based types are currently empty. In the future these functions may actually do something, and we will surely add more functions that are similarly patterned. Once everything has settled, we can make a table of "sub-driver" function pointers for each network type, and store a pointer to that table in the network object, then we can replace the switch statements with calls to functions in the table. The final step in this will be to add a new table (and corresponding new functions) for new network types as they are added. 2011-07-21 Laine Stump conf: support abstracted interface info in network XML The network XML is updated in the following ways: 1) The element can now contain a list of forward interfaces: The first of these takes the place of the dev attribute that is normally in - when defining a network you can specify either one, and on output both will be present. If you specify both on input, they must match. 2) In addition to forward modes of 'nat' and 'route', these new modes are supported: private, passthrough, vepa - when this network is referenced by a domain's interface, it will have the same effect as if the interface had been defined as type='direct', e.g.: "), then guest interfaces using this network will use libvirt's "host bridge" mode, equivalent to this: ... 3) A network can have multiple elements, which may be selected by the guest interface definition (by adding "portgroup='${name}'" in the element along with the network name). Currently a portgroup can only contain a virtportprofile, but the intent is that other configuration items may be put there int the future (e.g. bandwidth config). When building a guest's interface, if the XML itself has no virtportprofile, and if the requested network has a portgroup with a name matching the name given in the (or if one of the network's portgroups is marked with the "default='yes'" attribute), the virtportprofile from that portgroup will be used by the interface. 4) A network can have a virtportprofile defined at the top level, which will be used by a guest interface when connecting in one of the 'direct' modes if the guest interface XML itself hasn't specified any virtportprofile, and if there are also no matching portgroups on the network. 2011-07-21 Laine Stump conf: support abstracted interface info in domain interface XML the domain XML element is updated in the following ways: 1) can be specified when source type='network' (previously it was only valid for source type='direct') 2) A new attribute "portgroup" has been added to the element. When source type='network' (the only time portgroup is recognized), extra configuration information will be taken from the element of the given name in the network definition. 3) Each virDomainNetDef now also potentially has a virDomainActualNetDef which is a private object (never exported/imported via the public API, and not defined in the RNG) that is used to maintain information about the physical device that was actually used for a NetDef of type VIR_DOMAIN_NET_TYPE_NETWORK. The virDomainActualNetDef will only be parsed/formatted if the parse/format function is called with the VIR_DOMAIN_XML_INTERNAL_ACTUAL_NET flag set (which is only needed when saving/loading a running domain's state info to the stateDir). 2011-07-21 Laine Stump conf: virDomainNetDef points to (rather than contains) virtPortProfile The virtPortProfile in the domain interface struct is now a separately allocated object *pointed to by* (rather than contained in) the main virDomainNetDef object. This is done to make it easier to figure out when a virtualPortProfile has/hasn't been specified in a particular config. conf: put virtPortProfile struct / functions in a common location virtPortProfiles are currently only used in the domain XML, but will soon also be used in the network XML. To prepare for that change, this patch moves the structure definition into util/network.h and the parse and format functions into util/network.c (I decided that this was a better choice than macvtap.h/c for something that needed to always be available on all platforms). util: define MAX If util.h is going to have a MIN, it may as well also have MAX. 2011-07-21 Michal Privoznik destroy: Implement internal API for xenapi driver destroy: Implement internal API for xen driver destroy: Implement internal API for vmware driver destroy: Implement internal API for vbox driver destroy: Implement internal API for uml driver destroy: Implement internal API for phyp driver destroy: Implement internal API for openvz driver destroy: Implement internal API for lxc driver destroy: Implement internal API for libxl driver destroy: Implement internal API for ESX driver destroy: Implement internal API for qemu driver destroy: Wire up the remote protocol 2011-07-21 Michal Privoznik destroy: Define new public API virDomainDestroyFlags This introduces new API virDomainDestroyFlags to allow domain destroying with flags, as the existing API virDomainDestroy misses flags. The set of flags is defined in virDomainDestroyFlagsValues enum, which is currently commented, because it is empty. Calling this API with no flags set (@flags == 0) is equivalent calling virDomainDestroy. 2011-07-21 Eric Blake save: wire up remote protocol * src/remote/remote_driver.c (remote_driver): Add new callbacks. * src/remote/remote_protocol.x (remote_procedure): New RPCs. (remote_domain_save_flags_args, remote_domain_restore_flags_args): New structs. * src/remote_protocol-structs: Update. 2011-07-21 Eric Blake save: new public API to bypass file system cache on save/restore In order to choose whether to use O_DIRECT when saving a domain image to a file, we need a new flag. But virDomainSave was implemented before our policy of all new APIs having a flag argument. Likewise for virDomainRestore when restoring from a file. The new flag name is chosen as CACHE_BYPASS so as not to preclude a future solution that uses posix_fadvise once the Linux kernel has a smarter implementation of that interface. * include/libvirt/libvirt.h.in (virDomainCreateFlags) (virDomainCoreDumpFlags): Add a flag. (virDomainSaveFlags, virDomainRestoreFlags): New prototypes. * src/libvirt.c (virDomainSaveFlags, virDomainRestoreFlags): New API. * src/libvirt_public.syms: Export them. * src/driver.h (virDrvDomainSaveFlags, virDrvDomainRestoreFlags): New driver callbacks. 2011-07-21 Eric Blake qemu: fix error message with migrate2 xml Otherwise, an ABI mismatch gives error messages attributing the target xml string as current, and the current domain state as the new xml. * src/qemu/qemu_migration.c (qemuMigrationBegin): Use correct argument order. 2011-07-21 Eric Blake build: rename files.h to virfile.h In preparation for a future patch adding new virFile APIs. * src/util/files.h, src/util/files.c: Move... * src/util/virfile.h, src/util/virfile.c: ...here, and rename functions to virFile prefix. Macro names are intentionally left alone. * *.c: All '#include "files.h"' uses changed. * src/Makefile.am (UTIL_SOURCES): Reflect rename. * cfg.mk (exclude_file_name_regexp--sc_prohibit_close): Likewise. * src/libvirt_private.syms: Likewise. * docs/hacking.html.in: Likewise. * HACKING: Regenerate. 2011-07-21 Eric Blake command: avoid leaking fds across fork Since libvirt is multi-threaded, we should use FD_CLOEXEC as much as possible in the parent, and only relax fds to inherited after forking, to avoid leaking an fd created in one thread to a fork run in another thread. This gets us closer to that ideal, by making virCommand automatically clear FD_CLOEXEC on fds intended for the child, as well as avoiding a window of time with non-cloexec pipes created for capturing output. * src/util/command.c (virExecWithHook): Use CLOEXEC in parent. In child, guarantee that all fds to pass to child are inheritable. (getDevNull): Use CLOEXEC. (prepareStdFd): New helper function. (virCommandRun, virCommandRequireHandshake): Use pipe2. * src/qemu/qemu_command.c (qemuBuildCommandLine): Simplify caller. 2011-07-21 Eric Blake command: move all docs into .c file We already have a precedent of function documentation in C files, where it is closer to the implementation (witness libvirt.h vs. libvirt.c); maintaining docs in both files risks docs going stale. While I was at it, I used consistent doxygen style on all comments. * src/util/command.h: Remove duplicate docs, and move unique documentation... * src/util/command.c: ...here. Suggested by Matthias Bolte. 2011-07-21 Matthias Bolte rpc: Make the dispatch generator handle 'void name(void)' style procedures The only 'void name(void)' style procedure in the protocol is 'close' that is handled special, but also programming errors like a missing _args or _ret suffix on the structs in the .x files can create such a situation by accident. Making the generator aware of this avoids bogus errors from the generator such as: Use of uninitialized value in exists at ./rpc/gendispatch.pl line 967. Also this allows to get rid of the -c option and the special case code for the 'close' procedure, as the generator handles it now correctly. Reported by Michal Privoznik 2011-07-21 Eric Blake error: preserve errno when saving last error It is common to see the sequence: virErrorPtr save_err = virSaveLastError(); // do cleanup virSetError(save_err); virFreeError(save_err); on cleanup paths. But for functions where it is desirable to return the errno that caused failure, this sequence can clobber that errno. virFreeError was already safe; this makes the other two functions in the sequence safe as well, assuming all goes well (on OOM, errno will be clobbered, but then again, save_err won't reflect the real error that happened, so you are no longer preserving the real situation - that's life with OOM). * src/util/virterror.c (virSaveLastError, virSetError): Preserve errno. 2011-07-21 Matthias Bolte python: Fix makefile rule for code generation Commit 8665f85523f0451c changed generated.stamp to $(GENERATE).stamp, but missed one instance in the CLEANFILES list. This can break the build in case the generated code is deleted but the .stamp file stays around and therefore the code isn't regenerated. 2011-07-21 Daniel P. Berrange Fix uninitialized variable in QEMU CPU bandwidth code * src/qemu/qemu_driver.c: Fix uninitialized variable 2011-07-21 Wen Congyang fix make syntax-check error doc: Add documentation for new cputune elements period and quota We have added element period and quota. Document them in formatdomain.html.in. qemu: Implement cfs_period and cfs_quota's modification This patch implements cfs_period and cfs_quota's modification. We can use the command 'virsh schedinfo' to query or modify cfs_period and cfs_quota. If you query period or quota from config file, the value 0 means it does not set in the config file. If you set period or quota to config file, the value 0 means that delete current setting from config file. If you modify period or quota while vm is running, the value 0 means that use current value. qemu: Implement period and quota tunable XML configuration and parsing This patch implements period and quota tunable XML configuration and parsing. A quota or period of zero will be simply ignored. Update XML Schema for new entries Define the element cputune's child elements 'period' and 'quota': 100000 50000 cgroup: Implement cpu.cfs_period_us and cpu.cfs_quota_us tuning API This patch provides 4 APIs to get and set cpu.cfs_period_us and cpu.cfs_quota_us. Introduce the function virCgroupForVcpu Introduce the function virCgroupForVcpu() to create sub directory for each vcpu. 2011-07-21 Lai Jiangshan qemu: send-key: Implement the driver methods qemu driver just accept xt_kbd codeset's keycode, so the lib virtkey is used for translating keycodes from other codesets send-key: Expose the new API in virsh Also support string names for the linux keycode(auto detect) * tools/virsh.c: add new command "send-key" * tools/virsh.pod: documents the new command 2011-07-21 Lai Jiangshan util: add virtkeycode module Add virtkey lib for usage-improvment and keycode translating. Add 4 internal API for the aim const char *virKeycodeSetTypeToString(int codeset); int virKeycodeSetTypeFromString(const char *name); int virKeycodeValueFromString(virKeycodeSet codeset, const char *keyname); int virKeycodeValueTranslate(virKeycodeSet from_codeset, virKeycodeSet to_offset, int key_value); * include/libvirt/libvirt.h.in: extend virKeycodeSet enum * src/Makefile.am: add new virtkeycode module and rule to generate virkeymaps.h * src/util/virkeycode.c src/util/virkeycode.h: new module * src/util/virkeycode-mapgen.py: python generator for virkeymaps.h out of keymaps.csv * src/libvirt_private.syms: extend private symbols for new module * .gitignore: add generated virkeymaps.h 2011-07-21 Lai Jiangshan util: Add keymaps.csv Should keep it as the same as: http://git.gnome.org/browse/gtk-vnc/commit/src/keymaps.csv All master keymaps are defined in a CSV file. THis covers Linux keycodes, OSX keycodes, AT set1, 2 & 3, XT keycodes, the XT encoding used by the Linux KBD driver, USB keycodes, Win32 keycodes, the XT encoding used by Xorg on Cygwin, the XT encoding used by Xorg on Linux with kbd driver. * src/Makefile.am: added to EXTRA_DIST * src/util/keymaps.csv: new file 2011-07-21 Eric Blake maint: fix typos on guaranteed * src/conf/domain_event.c (virDomainEventDispatch): Fix typo. * src/internal.h (ATTRIBUTE_FMT_PRINTF): Likewise. * src/libvirt.c (virStreamEventUpdateCallback): Likewise. * src/remote/remote_driver.c (doRemoteOpen): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendProbeTarget): Likewise. * src/util/virterror.c (virConnCopyLastError, virCopyLastError): Likewise. * src/xen/xend_internal.h (xend_wait_for_devices): Likewise. 2011-07-20 Cole Robinson rpc: Pass through DISPLAY so ssh can launch askpass Though we prefer users to have SSH keys setup, virt-manager users still depend on remote SSH connections to launch a password dialog. This fixes launch ssh-askpass Fix suggested by danpb 2011-07-20 Michal Privoznik sysinfo: Don't try to run dmidecode on archs missing it DMI table is Intel & Intel-compatible specific. Therefore other architectures miss dmidecode command. So we always fail in searching for that command on non-Intel architectures. 2011-07-20 Guannan Ren Don't try to close a NULL virNetClientPtr * src/rpc/virnetclient.c: Skip close attempt if virNetClientPtr is NULL 2011-07-20 Daniel P. Berrange Honour key usage/purpose criticality flag If a key purpose or usage field is marked as non-critical in the certificate, then a data mismatch is not (ordinarily) a cause for rejecting the connection * src/rpc/virnettlscontext.c: Honour key usage/purpose criticality 2011-07-20 Daniel P. Berrange Fix checking of key usage/purpose data If key usage or purpose data is not present in the cert, the RFC recommends that access be allowed. Also fix checking of key usage to include requirements for client/server certs, and fix key purpose checking to treat data as a list of bits Fix mixed up error messages when reporting TLS certificate problems * src/rpc/virnettlscontext.c: Fix mixed up error messages 2011-07-20 Michal Privoznik udev: Don't try to dump DMI on non-intel archs DMI is Intel & Intel-compatible specific. Don't try to dump information on non-compatible architectures, which results only in error message in logs. 2011-07-20 Eric Blake build: fix broken build * src/libxl/libxl_driver.c (libxlDomainUndefineFlags): Use correct enum value. * src/remote_protocol-structs (remote_procedure): Likewise. 2011-07-20 Osier Yang undefine: Extend virsh undefine to support the new flag If the domain has managed save image, and --managed-save is not specified, then it fails with an error telling the user that a managed save image still exists. If the domain has managed save image, and --managed-save is specified, it invokes virDomainUndefineFlags. If virDomainUndefineFlags fails, then it tries to remove the managed save image using virDomainManagedSaveRemove first, with invoking virDomainUndefine following. (For compatibility between new virsh with this patch and older libvirt without this patch). Similarly if the domain has no managed save image. See the codes for detail. NOTE: Have not removing the codes checking if the domain is running in function "cmdUndefine", it will go along with qemu driver's fix (allow to undefine a running domain). 2011-07-20 Osier Yang undefine: Implement undefineFlags for all other drivers undefine: Implement internal API for libxl driver * src/libxl/libxl_driver.c: New callback for libxl_driver, new function libxlDomainUndefineFlags, and changes libxlDomainUndefine as a wrapper of libxlDomainUndefineFlags. undefine: Implement internal API for qemu driver * src/qemu/qemu_driver.c: New call back for qemu_driver, New function qemudDomainUndefineFlags, and changes on qemudDomainUndefine. undefine: Wire up the remote protocol 2011-07-20 Osier Yang undefine: Define the new API This introduces a new API virDomainUndefineFlags to control the domain undefine process, as the existing API virDomainUndefine doesn't support flags. Currently only flag VIR_DOMAIN_UNDEFINE_MANAGED_SAVE is supported. If the domain has a managed save image, including VIR_DOMAIN_UNDEFINE_MANAGED_SAVE in @flags will also remove that file, and omitting the flag will cause undefine process to fail. This patch also changes the behavior of virDomainUndefine, if the domain has a managed save image, the undefine will be refused. 2011-07-19 Eric Blake maint: fix spelling of Red Hat * cfg.mk (sc_copyright_format): Add to rule. * src/util/files.h: Fix offenders. * src/util/files.c: Likewise. 2011-07-19 Daniel P. Berrange Add sanity checking of basic constraints, key purpose & key usage Gnutls requires that certificates have basic constraints present to be used as a CA certificate. OpenSSL doesn't add this data by default, so add a sanity check to catch this situation. Also validate that the key usage and key purpose constraints contain correct data * src/rpc/virnettlscontext.c: Add sanity checking of certificate constraints 2011-07-19 Daniel P. Berrange Add some basic sanity checking of certificates before use If the libvirt daemon or libvirt client is configured with bogus certificates, it is very unhelpful to only find out about this when a TLS connection is actually attempted. Not least because the error messages you get back for failures are incredibly obscure. This adds some basic sanity checking of certificates at the time the virNetTLSContext object is created. This is at libvirt startup, or when creating a virNetClient instance. This checks that the certificate expiry/start dates are valid and that the certificate is actually signed by the CA that is loaded. * src/rpc/virnettlscontext.c: Add certificate sanity checks 2011-07-19 Daniel P. Berrange Fix reporting of errors for p2p migration Starting/ending jobs when closing the connection may reset any error which was reported earlier in p2p migration. We must save the original error before doing so. This means we can also just call virConnectClose as normal, instead of virUnrefConnect * src/qemu/qemu_migration.c: Preserve errors in p2p migration 2011-07-19 Daniel P. Berrange Add an explicit virNetClientClose method Since the I/O callback registered against virNetSocket will hold a reference on the virNetClient, we can't rely on the virNetClientFree to be able to close the network connection. The last reference will only go away when the event callback fires (likely due to EOF from the server). This is sub-optimal and can potentially cause a leak of the virNetClient object if the server were to not explicitly close the socket itself * src/remote/remote_driver.c: Explicitly close the client object when disconnecting * src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add a virNetClientClose method 2011-07-19 Daniel P. Berrange Use a virFreeCallback on virNetSocket to ensure safe release When unregistering an I/O callback from a virNetSocket object, there is still a chance that an event may come in on the callback. In this case it is possible that the virNetSocket might have been freed already. Make use of a virFreeCallback when registering the I/O callbacks and hold a reference for the entire time the callback is set. * src/rpc/virnetsocket.c: Register a free function for the file handle watch * src/rpc/virnetsocket.h, src/rpc/virnetserverservice.c, src/rpc/virnetserverclient.c, src/rpc/virnetclient.c: Add a free function for the socket I/O watches 2011-07-19 Daniel P. Berrange Add mutex locking and reference counting to virNetSocket Remove the need for a virNetSocket object to be protected by locks from the object using it, by introducing its own native locking and reference counting * src/rpc/virnetsocket.c: Add locking & reference counting 2011-07-19 Daniel P. Berrange Add some debugging for virNetClient reference counting * src/rpc/virnetclient.c: Add debugging of ref counts 2011-07-19 Eric Blake build: detect doc generation failure Commit 8665f85 introduced a slight regression in doc generation, since make only quits a rule on the first failed command ending with a newline rather than a semicolon. * docs/Makefile.am (html/index.html): Don't use xmllint unless xsltproc succeeded. * .gitignore: Ignore recently updated stamp file name. 2011-07-19 Alex Jia util: avoid fds leak when virEventPollAddHandle fail * src/util/event_poll.c: avoid file descriptors leak when virEventPollAddHandle fail on virEventPollInit function. 2011-07-19 Eric Blake Revert "virsh: make migrate --tunnelled imply --p2p" This reverts commit 40143fb697c3cb042197632c0286e7dadf40afd4. The patch prevents future compatibility if migration ever learns how to do tunnelled without p2p. 2011-07-19 Daniel P. Berrange Unregister event callback if a fatal error occurs during dispatch If we get an I/O error in the async event callback for an RPC client, we might not have consumed all pending data off the wire. This could result in the callback being immediately invoked again. At which point the same I/O might occur. And we're invoked again. And again...And again... Unregistering the async event callback if an error occurs is a good safety net. The real error will be seen when the next RPC method is invoked * src/rpc/virnetclient.c: Unregister event callback on error 2011-07-19 Daniel P. Berrange Quieten build & ensure API build scripts exit with non-zero status The current API build scripts will continue and exit with a zero status even if they find problems. This has been the cause of many build problems, or hidden build errors, in the past. Change the scripts so they always exit with a non-zero status for any problems they do not understand. Also turn off all debug output by default so they respect $(AM_V_GEN) * docs/Makefile.am: Use $(AM_V_GEN) for API/HTML scripts * docs/apibuild.py, python/generator.py: Exit with non-zero status if problems are found. Also be silent, not outputting any debug messages. * src/Makefile.am: Use $(AM_V_GEN) for ESX generator * python/Makefile.am: Tweak rule 2011-07-19 Osier Yang tests: Fix virshtest failure after dominfo changed Caused by the new changed "dominfo" command. virsh: Extend virsh dominfo to display if managed save image exists * tools/virsh.c: new column "Managed save" for "cmdDominfo". * tools/virsh.pod: Update document of "managedsave" to tell one can use "dominfo" to query whether a domain has any managed save image. doc: Correct documents for iface commands The problems: * Duplicate documents for "iface-name" * Lacks of document for "iface-mac" * Inconsistent option names with virsh help strings. 2011-07-18 Eric Blake virsh: make migrate --tunnelled imply --p2p We can make the virsh migrate UI friendlier by supplying the missing bit automatically instead of erroring out when requesting --tunnelled without --p2p. * tools/virsh.c (doMigrate): Make --p2p optional when using --tunnelled. * tools/virsh.pod (migrate): Tweak wording accordingly. 2011-07-18 Eric Blake libvirt: do not mix internal flags into public API There were two API in driver.c that were silently masking flags bits prior to calling out to the drivers, and several others that were explicitly masking flags bits. This is not forward-compatible - if we ever have that many flags in the future, then talking to an old server that masks out the flags would be indistinguishable from talking to a new server that can honor the flag. In general, libvirt.c should forward _all_ flags on to drivers, and only the drivers should reject unknown flags. In the case of virDrvSecretGetValue, the solution is to separate the internal driver callback function to have two parameters instead of one, with only one parameter affected by the public API. In the case of virDomainGetXMLDesc, it turns out that no one was ever mixing VIR_DOMAIN_XML_INTERNAL_STATUS with the dumpxml path in the first place; that internal flag was only used in saving and restoring state files, which happened to be in functions internal to a single file, so there is no mixing of the internal flag with a public flags argument. Additionally, virDomainMemoryStats passed a flags argument over RPC, but not to the driver. * src/driver.h (VIR_DOMAIN_XML_FLAGS_MASK) (VIR_SECRET_GET_VALUE_FLAGS_MASK): Delete. (virDrvSecretGetValue): Separate out internal flags. (virDrvDomainMemoryStats): Provide missing flags argument. * src/driver.c (verify): Drop unused check. * src/conf/domain_conf.h (virDomainObjParseFile): Delete declaration. (virDomainXMLInternalFlags): Move... * src/conf/domain_conf.c: ...here. Delete redundant include. (virDomainObjParseFile): Make static. * src/libvirt.c (virDomainGetXMLDesc, virSecretGetValue): Update clients. (virDomainMemoryPeek, virInterfaceGetXMLDesc) (virDomainMemoryStats, virDomainBlockPeek, virNetworkGetXMLDesc) (virStoragePoolGetXMLDesc, virStorageVolGetXMLDesc) (virNodeNumOfDevices, virNodeListDevices, virNWFilterGetXMLDesc): Don't mask unknown flags. * src/interface/netcf_driver.c (interfaceGetXMLDesc): Reject unknown flags. * src/secret/secret_driver.c (secretGetValue): Update clients. * src/remote/remote_driver.c (remoteSecretGetValue) (remoteDomainMemoryStats): Likewise. * src/qemu/qemu_process.c (qemuProcessGetVolumeQcowPassphrase): Likewise. * src/qemu/qemu_driver.c (qemudDomainMemoryStats): Likewise. * daemon/remote.c (remoteDispatchDomainMemoryStats): Likewise. 2011-07-18 Alex Jia virsh: avoid double free of domain * tools/virsh.c: avoid double free of domain, when weight value of blkiotune less than 0, codes will free domain and jump to cleanup section, however, cleanup will free domain again. Detected in valgrind run: ==21297== ERROR SUMMARY: 20 errors from 20 contexts (suppressed: 69 from 8) ==21297== ==21297== 1 errors in context 1 of 20: ==21297== Invalid read of size 4 ==21297== at 0x40E209B: virDomainFree (libvirt.c:2096) ==21297== by 0x8065274: cmdBlkiotune (virsh.c:3695) ==21297== by 0x8054CC1: vshCommandRun (virsh.c:13135) ==21297== by 0x806B967: main (virsh.c:14487) ==21297== Address 0x446ad48 is 0 bytes inside a block of size 36 free'd ==21297== at 0x4005B0A: free (vg_replace_malloc.c:325) ==21297== by 0x406814D: virFree (memory.c:310) ==21297== by 0x40D6635: virReleaseDomain (datatypes.c:243) ==21297== by 0x40D6C5E: virUnrefDomain (datatypes.c:280) ==21297== by 0x40E20B9: virDomainFree (libvirt.c:2101) ==21297== by 0x8065297: cmdBlkiotune (virsh.c:3613) ==21297== by 0x8054CC1: vshCommandRun (virsh.c:13135) ==21297== by 0x806B967: main (virsh.c:14487) ==21297== ==21297== ==21297== 1 errors in context 2 of 20: ==21297== Invalid read of size 4 ==21297== at 0x40E1FE6: virDomainFree (libvirt.c:2092) ==21297== by 0x8065274: cmdBlkiotune (virsh.c:3695) ==21297== by 0x8054CC1: vshCommandRun (virsh.c:13135) ==21297== by 0x806B967: main (virsh.c:14487) ==21297== Address 0x446ad48 is 0 bytes inside a block of size 36 free'd ==21297== at 0x4005B0A: free (vg_replace_malloc.c:325) ==21297== by 0x406814D: virFree (memory.c:310) ==21297== by 0x40D6635: virReleaseDomain (datatypes.c:243) ==21297== by 0x40D6C5E: virUnrefDomain (datatypes.c:280) ==21297== by 0x40E20B9: virDomainFree (libvirt.c:2101) ==21297== by 0x8065297: cmdBlkiotune (virsh.c:3613) ==21297== by 0x8054CC1: vshCommandRun (virsh.c:13135) ==21297== by 0x806B967: main (virsh.c:14487) * how to reproduce? % valgrind -v --leak-check=full virsh blkiotune guestname --weight -1 2011-07-18 Eric Blake docs: improve virsh man page synopses "optional" is not a very good meta-syntactic construct in our man page. I scrubbed this, and additionally improved some documentation on mutually exclusive options. For example, [[--live] [--config] | [--current]] implies a set of optional flags, where within the set you can have either --current or a choice of 0, 1, or both --live and --config. * tools/virsh.pod: Use "[name]" rather than "optional name" for optional arguments. 2011-07-18 Jiri Denemark build: Fix protocol-structs check in VPATH build $@ already included $(srcdir) 2011-07-18 Daniel P. Berrange Fix now dead cleanup of VMs on libvirtd restart When libvirtd restarts it will attempt to reconnect to existing LXC containers. If it loads a XML state file for the container the container will appear running. If we fail to read the PID file, or fail to connect to the LXC monitor, we should be killing off the guest, but if the VMs cgroup does not exist any more, cleanup will get skipped. Reading the PID file is also pointless since the PID is in the XML statefile In lxcReconnectVM we do not need to read the PID file. If part of the reconnect process fails we need to run the VM terminate code as a safety net. In lxcVMTerminate, if we can't obtain the VM cgroup, we know the process has died, but we must still run lxcVMCleanup to clear out the virDomainObjPtr live state * src/lxc/lxc_driver.c: Fix cleanup of dead VMs on restart 2011-07-18 Michal Privoznik docs: Fix spice documentation typo We missed ending tag for paragraph element 2011-07-17 Osier Yang rpc: Fix typos in rpc generator scripts These typos are introduced by file renaming in commit b17b4afaf. src/remote/qemu_protocol.x \ src/remote/remote_protocol.x \ src/rpc/gendispatch.pl: s/remote_generator/gendispatch/ src/rpc/genprotocol.pl: s/remote\/remote_protocol/remote_protocol/ 2011-07-16 Osier Yang qemu: Fix a regression of attaching device The regression is introduced by Commit da1eba6b, the new codes with this commit doesn't reset "ret" to "-1" when it fails on parsing the device XML (live device attachment) This patch changes the codes to reset the "ret" and "-1", and also changes the codes so that it don't modify "ret" for condition checking. How to reproduce: % cat test.xml % virsh attach-device $domain test.xml Device attached successfully The device attachment failed actually with error "unknown disk type 'oops'", however, it reports success. 2011-07-16 Eric Blake build: also track RPC on-wire enum values As long as we guarantee RPC struct layout stability, we might as well also guarantee RPC enum value constancy. * src/Makefile.am (r1, r2, PDWTAGS): Adjust rule to pick up named and anonymous enums. * src/remote_protocol-structs: Add enum values. * src/qemu_protocol-structs: Likewise. * src/virnetprotocol-structs: Likewise. 2011-07-16 Eric Blake docs: document dxml argument to migrate2 Commit 135554166 introduced a nice feature without documenting it. * src/libvirt.c (virDomainMigrate2): Add paragraph. 2011-07-16 Eric Blake build: add syntax check for proper flags use Enforce the recent flags cleanups - we want to use 'unsigned int flags' in any of our APIs (except where backwards compatibility is important, in the public migration APIs), and that all flags are checked for validity (except when there are stub functions that completely ignore the flags argument). There are a few minor tweaks done here to avoid false positives: signed arguments passed to open() are renamed oflags, and flags arguments that are legitimately ignored are renamed flags_unused. * cfg.mk (sc_flags_usage): New rule. (exclude_file_name_regexp--sc_flags_usage): And a few exemptions. (sc_flags_debug): Tweak wording. * src/util/iohelper.c (runIO, main): Rename variable. * src/util/util.c (virSetInherit): Likewise. * src/fdstream.h (virFDStreamOpenFile, virFDStreamCreateFile): Likewise. * src/fdstream.c (virFDStreamOpenFileInternal) (virFDStreamOpenFile, virFDStreamCreateFile): Likewise. * src/util/command.c (virExecWithHook) [WIN32]: Likewise. * src/util/util.c (virFileOpenAs, virDirCreate) [WIN32]: Likewise. * src/locking/lock_manager.c (virLockManagerPluginNew) [!HAVE_DLFCN_H]: Likewise. * src/locking/lock_driver_nop.c (virLockManagerNopNew) (virLockManagerNopAddResource, virLockManagerNopAcquire) (virLockManagerNopRelease, virLockManagerNopInquire): Likewise. 2011-07-16 Eric Blake xen: reject unknown flags Also fix a logic bug in xenXMDomain{Attach,Detach}DeviceFlags, where (flags & VIR_DOMAIN_DEVICE_MODIFY_CURRENT) is always false. * src/xen/xen_driver.c (xenUnifiedDomainXMLFromNative) (xenUnifiedDomainXMLToNative, xenUnifiedDomainBlockPeek): Reject unknown flags. * src/xen/xen_hypervisor.c (xenHypervisorOpen) (xenHypervisorGetDomainState): Likewise. * src/xen/xen_inotify.c (xenInotifyOpen): Likewise. * src/xen/xs_internal.c (xenStoreOpen, xenStoreDomainGetState) (xenStoreDomainReboot): Likewise. * src/xen/xend_internal.c (xenDaemonOpen, xenDaemonDomainReboot) (xenDaemonDomainCoreDump, xenDaemonDomainGetState) (xenDaemonDomainMigratePrepare, xenDaemonDomainSetVcpusFlags, xenDaemonDomainGetVcpusFlags, xenDaemonAttachDeviceFlags, xenDaemonDetachDeviceFlags): Likewise. (xenDaemonDomainGetXMLDesc): Prefer unsigned flags. * src/xen/xend_internal.h (xenDaemonDomainGetXMLDesc): Likewise. * src/xen/xm_internal.h (xenXMDomainGetXMLDesc): Likewise. * src/xen/xm_internal.c (xenXMDomainGetXMLDesc): Likewise. (xenXMOpen, xenXMDomainGetState, xenXMDomainSetVcpusFlags) (xenXMDomainGetVcpusFlags): Reject unknown flags. (xenXMDomainAttachDeviceFlags, xenXMDomainDetachDeviceFlags): Likewise, and avoid always-false conditional. * src/xen/xen_driver.h (XEN_MIGRATION_FLAGS): New define. 2011-07-16 Eric Blake esx: reject unknown flags Silently ignored flags get in the way of new features that use those flags. Regarding ESX migration flags - right now, ESX silently enforces VIR_MIGRATE_PERSIST_DEST, VIR_MIGRATE_UNDEFINE_SOURCE, and VIR_MIGRATE_LIVE, even if those flags were not supplied; it ignored other flags. This patch does not change the implied bits (it permits but does not require them), but enforces only the supported bits. If further cleanup is needed to be more particular about migration flags, that should be a separate patch. * src/esx/esx_device_monitor.c (esxDeviceOpen): Reject unknown flags. * src/esx/esx_driver.c (esxOpen, esxDomainReboot) (esxDomainXMLFromNative, esxDomainXMLToNative) (esxDomainMigratePrepare, esxDomainMigratePerform) (esxDomainMigrateFinish): Likewise. * src/esx/esx_interface_driver.c (esxInterfaceOpen): Likewise. * src/esx/esx_network_driver.c (esxNetworkOpen): Likewise. * src/esx/esx_nwfilter_driver.c (esxNWFilterOpen): Likewise. * src/esx/esx_secret_driver.c (esxSecretOpen): Likewise. * src/esx/esx_storage_driver.c (esxStorageOpen): Likewise. 2011-07-15 Eric Blake virsh: improve option handling The documentation for vshCommandOptString claims that it returns -1 on a missing required argument, but in reality, that error message was unreachable (it was buried inside an if clause that is true only if the argument was present). The code was so hairy that I decided a rewrite would make it easier to understand, and actually return the error values we want. Meanwhile, our construction guarantees that all vshCmdOpt have a non-null def member, so there are some redundant checks that can be trimmed. * tools/virsh.c (vshCommandOpt): Alter signature. (vshCommandOptInt, vshCommandOptUInt, vshCommandOptUL) (vshCommandOptString, vshCommandOptLongLong) (vshCommandOptULongLong, vshCommandOptBool): Adjust all callers. (vshCommandOptArgv): Remove dead condition. 2011-07-15 Eric Blake flags: fix domain_conf migration regression Commit 461e0f1a broke migration, because there was a code path that tried to enable an internal flag while still going through the public function. Split the internal flag into a separate callback, and validate that flags do not overlap. * src/conf/domain_conf.c (virDomainDefFormat): Split... (virDomainDefFormatInternal): ...to separate the flag check. (virDomainObjFormat): Adjust caller. 2011-07-15 Dave Allan website: Point main page links to libvirt driver pages The "libvirt supports:" section on the main page of libvirt.org contains a list of hypervisors with links that point to the sites of the underlying virt technologies. The entry for KVM points to http://www.linux-kvm.org/, for example. People coming to libvirt.org for the first time are likely to know about those sites, and they're probably interested in how libvirt manages those technologies. This patch points those links to the libvirt driver pages instead. It also consolidates KVM and QEMU as there is only one libvirt driver page for them. Finally, it adds a line about networking support. v2: incorporate Eric's feedback adding project links to driver pages. website: Add project links to KVM/QEMU driver page website: Add project links to Xen driver page website: Add project links to LXC driver page website: Add project links to OpenVZ driver page website: Add project links to UML driver page website: Add project links to Virtualbox driver page website: Add project links to ESX driver page website: Add project links to VMware driver page 2011-07-15 Eric Blake flags: fix qemu migration regression Commit f548480b broke migration v3 on qemu, because the driver passed flags on through to qemu_migration even though qemu_migration wasn't using those flags. * src/qemu/qemu_migration.h (QEMU_MIGRATION_FLAGS): New define. * src/qemu/qemu_driver.c: Simplify all migration callbacks. * src/qemu/qemu_migration.c (qemuMigrationConfirm): Fix regression. 2011-07-15 Eric Blake flags: use common dumpxml flags check The previous patches only cleaned up ATTRIBUTE_UNUSED flags cases; auditing the drivers found other places where flags was being used but not validated. In particular, domainGetXMLDesc had issues with clients accepting a different set of flags than the common virDomainDefFormat helper function. * src/conf/domain_conf.c (virDomainDefFormat): Add common flag check. * src/uml/uml_driver.c (umlDomainAttachDeviceFlags) (umlDomainDetachDeviceFlags): Reject unknown flags. * src/vbox/vbox_tmpl.c (vboxDomainGetXMLDesc) (vboxDomainAttachDeviceFlags) (vboxDomainDetachDeviceFlags): Likewise. * src/qemu/qemu_driver.c (qemudDomainMemoryPeek): Likewise. (qemuDomainGetXMLDesc): Document common flag handling. * src/libxl/libxl_driver.c (libxlDomainGetXMLDesc): Likewise. * src/lxc/lxc_driver.c (lxcDomainGetXMLDesc): Likewise. * src/openvz/openvz_driver.c (openvzDomainGetXMLDesc): Likewise. * src/phyp/phyp_driver.c (phypDomainGetXMLDesc): Likewise. * src/test/test_driver.c (testDomainGetXMLDesc): Likewise. * src/vmware/vmware_driver.c (vmwareDomainGetXMLDesc): Likewise. * src/xenapi/xenapi_driver.c (xenapiDomainGetXMLDesc): Likewise. 2011-07-15 Eric Blake maint: fix typos Our XML prefers "shareable" over "sharable". * docs/internals/locking.html.in: s/sharable/shareable/ * src/locking/lock_driver_sanlock.c (virLockManagerSanlockAddResource): Likewise. 2011-07-15 Hu Tao add --cache, --serial, --shareable and --address to attach-disk This adds four options for virsh command attach-disk. --cache option allows user to specify cache mode of disk device from virsh command line when attaching a disk device. --serial option allows user to specify serial string of disk device from virsh command line when attaching a disk device. --shareable option allows user to specify whether the disk device is shareable between domains when attaching a disk device from virsh command line. --address option allows user to specify address of disk device when attaching a disk device. 2011-07-15 Daniel P. Berrange Fix error message for missing TLS write function * src/rpc/virnettlscontext.c: s/read/write/ 2011-07-15 Daniel P. Berrange Fix reporting of cert validation failures If the server succesfully validates the client cert, it will send back a single byte, under TLS. If it fails, it will close the connection. In this case, we were just reporting the standard I/O error. The original RPC code had a special case hack for the GNUTLS_E_UNEXPECTED_PACKET_LENGTH error code to make us report a more useful error message * src/rpc/virnetclient.c: Return ENOMSG if we get GNUTLS_E_UNEXPECTED_PACKET_LENGTH * src/rpc/virnettlscontext.c: Report cert failure if we see ENOMSG 2011-07-15 Dave Allan Fix error code for storage operations Many volume operations will fail if the volume in question is being allocated. These operations were returning VIR_ERR_INTERNAL_ERROR when they should be returning VIR_ERR_OPERATION_INVALID. 2011-07-15 Taku Izumi setvcpus: add "--current" option to "virsh setvcpus" This patch adds the --current option to "virsh setvcpus" command. Currently "virsh setvcpus" command supports "--live" and "--config" , but "--current" option. From view of consistency, it's reasonable to support "--current" option too. When --current is specified, it affects a "current" domain. 2011-07-15 Taku Izumi setvcpus: extend qemuDomainSetVcpusFlags() to support current flag This patch extends qemudDomainSetVcpusFlags() function to support VIR_DOMAIN_AFFECT_CURRENT flag. 2011-07-15 Taku Izumi setvcpus: extend virDomainSetVcpusFlags API to support current flag This patch extends virDomainSetVcpusFlags API to support VIR_DOMAIN_AFFECT_CURRENT flag. Now because most APIs accept VIR_DOMAIN_AFFECT_CURRENT flags, virDomainSetVcpusFlags API should also do. 2011-07-15 Daniel P. Berrange Add domain events support to UML driver * src/uml_conf.h: Add queue for dispatch of domain events * src/uml_driver.c: Trigger domain events upon important lifecycle transitions 2011-07-14 Eric Blake docs: mention EMOTIVE as a libvirt-using app * docs/apps.html.in: Add EMOTIVE. Reported by Alex Vaqué Brull . 2011-07-14 Eric Blake virnetsocket: use new API for uniform child cleanup Rather than trying to clean up the ssh child ourselves, and risk subtle differences from the socket creation error path, we can just use the new APIs. * src/rpc/virnetsocket.c (virNetSocketFree): Use new function. 2011-07-14 Eric Blake fdstream: avoid child process leak on error By requesting the pid in virCommandRunAsync, fdstream was claiming that it would manually wait for the process. But on the failure path, the child process was being leaked. * src/fdstream.c (virFDStreamOpenFileInternal): Auto-reap child. 2011-07-14 Eric Blake command: introduce virPidWait, virPidAbort When using virCommandRunAsync and saving the pid for later, it is useful to be able to reap that pid in the same way that it would have been auto-reaped by virCommand if we had passed NULL for the pid argument in the first place. * src/util/command.c (virPidWait, virPidAbort): New functions, created from... (virCommandWait, virCommandAbort): ...bodies of these. (includes): Drop duplicate . Ensure that our pid_t assumptions hold. (virCommandRunAsync): Improve documentation. * src/util/command.h (virPidWait, virPidAbort): New prototypes. * src/libvirt_private.syms: Export them. * docs/internals/command.html.in: Document them. 2011-07-14 Jamie Strandboge update apparmor security driver for new udev paths In the Ubuntu development release we recently got a new udev that moves /var/run to /run, /var/lock to /run/lock and /dev/shm to /run/shm. This change in udev requires updating the apparmor security driver in libvirt[1]. Attached is a patch that: * adjusts src/security/virt-aa-helper.c to allow both LOCALSTATEDIR/run/libvirt/**/%s.pid and /run/libvirt/**/%s.pid. While the profile is not as precise, LOCALSTATEDIR/run/ is typically a symlink to /run/ anyway, so there is no additional access (remember that apparmor resolves symlinks, which is why this is still required even if /var/run points to /run). * adjusts example/apparmor/libvirt-qemu paths for /dev/shm [1]https://launchpad.net/bugs/810270 -- Jamie Strandboge | http://www.canonical.com 2011-07-14 Matthias Bolte xenapi: Improve error reporting in xenapiOpen Use better suited error code and avoid NULL in error messsage as *privP->session->error_description can be NULL. 2011-07-14 Eric Blake build: fix VPATH build of todo Without this patch, the make rule in a VPATH build was trying to invoke ../../docs/../../docs/todo.pl, which didn't exist. * docs/Makefile.am (todo.html.in): Using $< already implies $(srcdir) in GNU make VPATH situations. 2011-07-14 Eric Blake build: check for virnetprotocol on-the-wire stability Similar to the recent qemu_protocol-structs addition. * src/virnetprotocol-structs: New file. * src/Makefile.am (%_protocol-structs): Factor body... (PDWTAGS): ...into new helper macro. (virnetprotocol-structs): New rule. (PROTOCOL_STRUCTS): Add virnetprotocol-structs. 2011-07-14 Wen Congyang build: disable some drivers when building without libvirt daemon We disable some drivers when building without libvirtd in configure, but we do not do the same thing in libvirt.spec. It may break rpm building without libvirtd. 2011-07-14 Michal Privoznik storage: Avoid memory leak on metadata fetching Getting metadata on storage allocates a memory (path) which need to be freed after use otherwise it gets leaked. This means after use of virStorageFileGetMetadataFromFD or virStorageFileGetMetadata one must call virStorageFileFreeMetadata to free it. This function frees structure internals and structure itself. 2011-07-14 Jiri Denemark qemu: Don't overwrite errors by closefd in error paths When qemuMonitorCloseFileHandle is called in error path, we need to preserve the original error since a possible further error when running closefd monitor command is not very useful to users. qemu: Save domain status ASAP after creating qemu process When creating new qemu process we saved domain status XML only after the process was fully setup and running. In case libvirtd was killed before the whole process finished, once libvirtd started again it didn't know anything about the new process and we end up with an orphaned qemu process. Let's save the domain status XML as soon as we know the PID so that libvirtd can kill the process on restart. 2011-07-14 Eric Blake build: avoid ATTRIBUTE_UNUSED in headers The compiler might optimize based on our declaration that something is unused. Putting that declaration in the header risks getting out of sync with the actual implementation, so it belongs better only in the .c files. We were mostly compliant, and a new syntax check will help us in the future. * cfg.mk (sc_avoid_attribute_unused_in_header): New syntax check. * src/nodeinfo.h (nodeGetCPUStats, nodeGetMemoryStats): Delete attribute already present in .c file. * src/qemu/qemu_domain.h (qemuDomainEventFlush): Likewise. * src/util/virterror_internal.h (virReportErrorHelper): Parameters are actually used by .c file. * src/xenxs/xen_sxpr.h (xenFormatSxprDisk): Adjust prototype. * src/xenxs/xen_sxpr.c (xenFormatSxprDisk): Delete unused argument. (xenFormatSxpr): Adjust caller. * src/xen/xend_internal.c (xenDaemonAttachDeviceFlags) (xenDaemonUpdateDeviceFlags): Likewise. Suggested by Daniel Veillard. 2011-07-14 Eric Blake remote: prefer unsigned flags * src/remote/remote_driver.c (call, remoteOpenSecondaryDriver): Prefer unsigned flags. 2011-07-14 Eric Blake conf: delete unused flags arguments For static functions not used as callbacks, there's no need to keep an unused parameter. * src/conf/domain_conf.c (virDomainChrDefParseTargetXML) (virDomainTimerDefParseXML, virDomainHostdevSubsysUsbDefParseXML) (virDomainVcpuPinDefParseXML): Drop unused parameter. (virDomainChrDefParseXML, virDomainDefParseXML) (virDomainHostdevDefParseXML): Update callers. (virDomainNetDefParseXML): Mark flags used. 2011-07-14 Supriya Kannery virsh: Update virsh man page Valid loglevel range for virsh is 0-4. Update virsh man page accordingly. Also explain virsh ENV variables and values. 2011-07-14 Jiri Denemark virsh: fix previous patch The last patch breaks make check for two reasons. First, it reverses the condition but leaves default level unchanged, so instead of not printing anything but errors before the patch it now prints all debug messages by default. Second, you forgot to change -d5 option passed to virsh in tests/virsh-optparse to -d0; the script wants to see all debug messages. 2011-07-14 Supriya Kannery virsh: Make "DEBUG" loglevel the superset Aligning loglevel values of virsh to that of libvirt. "DEBUG"=0 loglevel, when specified through commandline or env variable, should log all the messages. "ERROR=4" should log only error messages. virsh: Avoid using magic numbers for logging Replace magic numbers with loglevel variables. 2011-07-14 Jiri Denemark util: Avoid duplicating virFileOpenAsNoFork in virFileOpenAs In 2f4d2496a88055a8343b3efca618522da8715d92 I didn't notice that one part of virFileOpenAs doesn't actually call to virFileOpenAsNoFork but rather includes a copy of the code from there. 2011-07-14 Eric Blake build: don't hand-roll cloexec code No need to repeat common code. * bootstrap.conf (gnulib_modules): Import calloc-posix. * src/util/bridge.c (brInit): Use virSetCloseExec. (brSetInterfaceUp): Adjust flags name. * src/uml/uml_driver.c (umlSetCloseExec): Delete. (umlStartVMDaemon): Use util version instead. 2011-07-14 Eric Blake conf: prefer unsigned flags * src/conf/cpu_conf.h (virCPUDefFormat, virCPUDefFormatBuf): Change flags type. * src/conf/cpu_conf.c (virCPUDefFormat, virCPUDefFormatBuf): Likewise. * src/conf/storage_conf.c (_virStoragePoolOptions): Likewise. * src/datatypes.h (_virConnect, _virStream): Likewise. python: prefer unsigned flags * python/libvirt-override.c (libvirt_virConnectOpenAuth) (libvirt_virDomainSnapshotListNames) (libvirt_virDomainRevertToSnapshot): Change flags type. 2011-07-14 Eric Blake node_device: avoid implicit int 'unsigned a' and 'unsigned int a' are synonyms, but we generally always spell out the 'int' in that case. Fixing this will avoid a false positive in the next syntax-check commit. * src/conf/node_device_conf.h (pci_config_address) (_virNodeDevCapsDef): Prefer 'unsigned int' over 'unsigned'. 2011-07-14 Eric Blake virsh, daemon: prefer unsigned flags * tools/virsh.c (vshCmdDef): Change flags type. * daemon/remote.c (remoteDispatchOpen): Likewise. xenapi: reject unknown flags * src/xenapi/xenapi_driver.c (xenapiOpen, xenapiDomainReboot): Reject unknown flags. (xenapiDomainGetXMLDesc): Likewise, and pass known flags through to XML generation. vmware: reject unknown flags * src/vmware/vmware_driver.c (vmwareOpen, vmwareDomainReboot) (vmwareDomainCreateXML, vmwareDomainCreateWithFlags): Reject unknown flags. vbox: reject unknown flags * src/vbox/vbox_driver.c (vboxOpenDummy): Reject unknown flags. * src/vbox/vbox_tmpl.c (vboxOpen, vboxDomainReboot) (vboxNetworkOpen, vboxNetworkGetXMLDesc, vboxStorageOpen) (vboxStorageVolCreateXML, vboxStorageVolDelete) (vboxStorageVolGetXMLDesc, vboxDomainScreenshot): Likewise. 2011-07-13 Eric Blake uml: reject unknown flags * src/uml/uml_driver.c (umlOpen, umlDomainGetXMLDesc) (umlDomainBlockPeek): Reject unknown flags. test: reject unknown flags * src/test/test_driver.c (testOpen, testDomainCoreDump) (testOpenNetwork, testNetworkGetXMLDesc, testOpenInterface) (testInterfaceChangeBegin, testInterfaceChangeCommit) (testInterfaceChangeRollback, testInterfaceGetXMLDesc) (testInterfaceDefineXML, testInterfaceCreate) (testInterfaceDestroy, testStorageOpen, testStoragePoolStart) (testStorageFindPoolSources, testStoragePoolCreate) (testStoragePoolDefine, testStoragePoolBuild) (testStoragePoolDelete, testStoragePoolRefresh) (testStoragePoolGetXMLDesc, testStorageVolumeCreateXML) (testStorageVolumeCreateXMLFrom, testStorageVolumeDelete) (testStorageVolumeGetXMLDesc, testDevMonOpen) (testNodeNumOfDevices, testNodeListDevices) (testNodeDeviceGetXMLDesc, testNodeDeviceCreateXML) (testSecretOpen, testNWFilterOpen): Reject unknown flags. qemu: reject unknown flags * src/qemu/qemu_driver.c (qemudOpen, qemuDomainScreenshot) (qemuDomainXMLFromNative, qemuDomainXMLToNative) (qemudDomainBlockPeek, qemuCPUCompare, qemuCPUBaseline): Reject unknown flags. * src/qemu/qemu_migration.c (qemuMigrationConfirm): Likewise. (_qemuMigrationCookie, qemuMigrationCookieXMLParse) (qemuMigrationCookieXMLParseStr, qemuMigrationBakeCookie) (qemuMigrationEatCookie): Make flags unsigned. * src/qemu/qemu_domain.h: (qemuDomainDefFormatXML) (qemuDomainFormatXML): Prefer unsigned flags. * src/qemu/qemu_domain.c (qemuDomainDefFormatXML) (qemuDomainFormatXML): Likewise. (qemuDomainOpenLogHelper, qemuDomainCreateLog): Rename variable. phyp: reject unknown flags * src/phyp/phyp_driver.c (phypOpen, phypDomainReboot) (phypVIOSDriverOpen): Reject unknown flags. openvz: reject unknown flags * src/openvz/openvz_driver.c (openvzDomainReboot, openvzOpen): Reject unknown flags. lxc: reject unknown flags * src/lxc/lxc_driver.c (lxcOpen, lxcDomainSetMemoryParameters) (lxcDomainGetMemoryParameters): Reject unknown flags. * src/lxc/lxc_container.c (lxcContainerStart): Rename flags to cflags to reflect that it is not tied to libvirt. libxl: reject unknown flags * src/libxl/libxl_driver.c (libxlOpen, libxlDomainReboot) (libxlDomainXMLFromNative, libxlDomainXMLToNative) (libxlDomainCreateWithFlags): Reject unknown flags. 2011-07-13 Nan Zhang virsh: fix missing prompt message for 'snapshot-delete' command Make the command 'virsh snapshot-delete' has the appropriate prompt message when executing sucessful or failed. 2011-07-13 Peter Krempa qemu: build failed due to unused variables While compiling on F15 build crashed (probably because of new GCC). 2011-07-13 Eric Blake storage: reject unknown flags * src/storage/storage_backend.c (virStorageBackendCreateBlockFrom) (virStorageBackendCreateQemuImg) (virStorageBackendCreateQcowCreate): Reject unknown flags. * src/storage/storage_backend_disk.c (virStorageBackendDiskBuildPool) (virStorageBackendDiskDeleteVol): Likewise. * src/storage/storage_backend_fs.c (virStorageBackendFileSystemNetFindPoolSources) (virStorageBackendFileSystemBuild) (virStorageBackendFileSystemDelete, createFileDir) (virStorageBackendFileSystemVolBuildFrom) (virStorageBackendFileSystemVolDelete): Likewise. * src/storage/storage_backend_iscsi.c (virStorageBackendISCSIFindPoolSources): Likewise. * src/storage/storage_backend_logical.c (virStorageBackendLogicalFindPoolSources) (virStorageBackendLogicalBuildPool) (virStorageBackendLogicalDeletePool) (virStorageBackendLogicalDeleteVol): Likewise. * src/storage/storage_driver.c (storageOpen, storagePoolCreate) (storagePoolDefine, storagePoolRefresh, storagePoolGetXMLDesc) (storageVolumeCreateXML, storageVolumeCreateXMLFrom) (storageVolumeGetXMLDesc): Likewise. node_device: reject unknown flags * src/node_device/node_device_driver.c (nodeNumOfDevices) (nodeListDevices, nodeDeviceGetXMLDesc, nodeDeviceCreateXML): Reject unknown flags. * src/node_device/node_device_hal.c (halNodeDrvOpen): Likewise. * src/node_device/node_device_udev.c (udevNodeDrvOpen): Likewise. 2011-07-13 Eric Blake util: reject unknown flags, and prefer unsigned flags Silently ignored flags get in the way of new features that use those flags. Also, an upcoming syntax check will favor unsigned flags. * src/nodeinfo.h (nodeGetCPUStats, nodeGetMemoryStats): Drop unused attribute. * src/interface/netcf_driver.c (interfaceOpenInterface) (interfaceDefineXML, interfaceCreate, interfaceDestroy): Reject unknown flags. * src/network/bridge_driver.c (networkOpenNetwork) (networkGetXMLDesc): Likewise. * src/nwfilter/nwfilter_driver.c (nwfilterOpen): Likewise. * src/secret/secret_driver.c (secretOpen, secretDefineXML) (secretGetXMLDesc, secretSetValue): Likewise. * src/util/logging.c (virLogDefineFilter, virLogDefineOutput) (virLogMessage): Likewise; also use unsigned flags. * src/util/logging.h (virLogDefineFilter, virLogDefineOutput) (virLogMessage): Change signature. * src/util/command.c (virExecWithHook): Likewise. 2011-07-13 Eric Blake libvirt-qemu: use unsigned flags Like commit 1740c381, but for libvirt-qemu. * src/remote/qemu_protocol.x (qemu_monitor_command_args): Adjust type to match API. * src/qemu_protocol-structs: Update accordingly. 2011-07-13 Eric Blake maint: print flags in hex during debug Continuation of commit 313ac7fd, and enforce things with a syntax check. Technically, virNetServerClientCalculateHandleMode is not printing a mode_t, but rather a collection of VIR_EVENT_HANDLE_* bits; however, these bits are < 8, so there is no different in the output, and that was the easiest way to silence the new syntax check. * cfg.mk (sc_flags_debug): New syntax check. (exclude_file_name_regexp--sc_flags_debug): Add exemptions. * src/fdstream.c (virFDStreamOpenFileInternal): Print flags in hex, mode_t in octal. * src/libvirt-qemu.c (virDomainQemuMonitorCommand) (virDomainQemuAttach): Likewise. * src/locking/lock_driver_nop.c (virLockManagerNopInit): Likewise. * src/locking/lock_driver_sanlock.c (virLockManagerSanlockInit): Likewise. * src/locking/lock_manager.c: Likewise. * src/qemu/qemu_migration.c: Likewise. * src/qemu/qemu_monitor.c: Likewise. * src/rpc/virnetserverclient.c (virNetServerClientCalculateHandleMode): Print mode with %o. 2011-07-13 Eric Blake maint: exclude more files from syntax check * cfg.mk (VC_LIST_ALWAYS_EXCLUDE_REGEX): Exempt docs/api_extension/*.patch. (exclude_file_name_regexp--sc_prohibit_always_true_header_tests) (exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF) (exclude_file_name_regexp--sc_prohibit_fork_wrappers) (exclude_file_name_regexp--sc_trailing_blank): Simplify. (exclude_file_name_regexp--sc_prohibit_gettext_noop): Delete. (exclude_file_name_regexp--sc_prohibit_close) (exclude_file_name_regexp--sc_prohibit_nonreentrant) (exclude_file_name_regexp--sc_prohibit_sprintf): Tighten. 2011-07-13 Eric Blake util: honor anchored names when searching for executables I got bit in a debugging session on an uninstalled libvirtd; the code tried to call out to the installed $LIBEXECDIR/libvirt_iohelper instead of my just-built version. So I set a breakpoint and altered the binary name to be "./src/libvirt_iohelper", and it still failed because I don't have "." on my PATH. According to POSIX, execvp only searches PATH if the name does not contain a slash. Since we are trying to mimic that behavior, an anchored name should be relative to the current working dir. This tightens existing behavior, but most callers already pass an absolute name or a name with no slashes, so it probably won't be noticeable. * src/util/util.c (virFindFileInPath): Anchored relative names do not invoke a PATH search. 2011-07-13 Daniel P. Berrange Ensure signal handler propagates fatal signals to default handler When replacing the default SEGV/ABORT/BUS signal handlers you can't rely on the process being terminated after your custom handler runs. It is neccessary to manually restore the default handler and then re-raise the signal * src/rpc/virnetserver.c: Restore default handler and raise signal 2011-07-13 Daniel P. Berrange Remove unused virNetServerProgramErrorHander typedef * src/rpc/virnetserverprogram.h: Remove unused typedef for virNetServerProgramErrorHander function callback * daemon/remote.h: Remove decl for non-existant variables 2011-07-13 Jiri Denemark qemu: Fix monitor unlocking in some error paths When monitor is entered with qemuDomainObjEnterMonitorWithDriver, the correct method for leaving and unlocking the monitor is qemuDomainObjExitMonitorWithDriver. 2011-07-13 Jiri Denemark qemu: Consolidate qemuMigrationPrepare{Direct,Tunnel} Most of the code in these two functions is supposed to be identical but currently it isn't (which is natural since the code is duplicated). Let's move common parts of these functions into qemuMigrationPrepareAny. This also fixes qemuMigrationPrepareTunnel which didn't store received lockState in the domain object. 2011-07-13 Jiri Denemark qemu: Add support for job phase Asynchronous jobs may take long time to finish and may consist of several phases which we need to now about to help with recovery/rollback after libvirtd restarts. qemu: Recover from interrupted jobs Detect and react on situations when libvirtd was restarted or killed when a job was active. qemu: Save job type in domain status XML If libvirtd is restarted when a job is running, the new libvirtd process needs to know about that to be able to recover and rollback the operation. 2011-07-13 Jiri Denemark qemu: Allow all query commands to be run during long jobs Query commands are safe to be called during long running jobs (such as migration). This patch makes them all work without the need to special-case every single one of them. The patch introduces new job.asyncCond condition and associated job.asyncJob which are dedicated to asynchronous (from qemu monitor point of view) jobs that can take arbitrarily long time to finish while qemu monitor is still usable for other commands. The existing job.active (and job.cond condition) is used all other synchronous jobs (including the commands run during async job). Locking schema is changed to use these two conditions. While asyncJob is active, only allowed set of synchronous jobs is allowed (the set can be different according to a particular asyncJob) so any method that communicates to qemu monitor needs to check if it is allowed to be executed during current asyncJob (if any). Once the check passes, the method needs to normally acquire job.cond to ensure no other command is running. Since domain object lock is released during that time, asyncJob could have been started in the meantime so the method needs to recheck the first condition. Then, normal jobs set job.active and asynchronous jobs set job.asyncJob and optionally change the list of allowed job groups. Since asynchronous jobs only set job.asyncJob, other allowed commands can still be run when domain object is unlocked (when communicating to remote libvirtd or sleeping). To protect its own internal synchronous commands, the asynchronous job needs to start a special nested job before entering qemu monitor. The nested job doesn't check asyncJob, it only acquires job.cond and sets job.active to block other jobs. 2011-07-12 Jiri Denemark qemu: Consolidate {Enter,Exit}Monitor{,WithDriver} EnterMonitor and ExitMonitor methods are very similar to their *WithDriver variants; consolidate them into EnterMonitorInternal and ExitMonitorInternal to avoid (mainly future) code duplication. qemu: Consolidate BeginJob{,WithDriver} into a single method This avoids code duplication and also avoids relying on good luck that ignore_value(virDomainObjUnref(obj)) doesn't set errno. qemu: Separate job related data into a new object 2011-07-12 Eric Blake maint: avoid incremental 'make syntax-check' failure Incrementally running 'make syntax-check' on a tree previously built after commit 62dee6f but before 44036460 fails sc_po_check (because the generated qemu_dispatch.h gained translatable strings). This is a followup to commit addaa537 for that scenario. * cfg.mk (sc_po_check): Add another prereq. ($(srcdir)/daemon/qemu_dispatch.h): Add rule. 2011-07-12 Daniel P. Berrange Skip bugs which are CLOSED in todo list * docs/todo.pl: Skip closed bugs 2011-07-12 Daniel P. Berrange Add missing cleanup for transient guests in UML driver The UML inotify handler would kill off guests when certain conditions arise, but it forgot to remove transient guests from the list of domains * src/uml/uml_driver.c: Cleanup transient guests 2011-07-12 Daniel P. Berrange Add auditing to the UML driver * src/uml/uml_driver.c: Add audit hooks Add auditing to the LXC driver * src/lxc/lxc_driver.c: Add auditing hooks 2011-07-12 Daniel P. Berrange Add virtualization type in audit messages Since a host can run several different virtualization types at the same time, audit messages should allow domains to be identified. Add a 'virt={qemu,kvm,uml,lxc,...}' key to domain audit messages * src/conf/domain_audit.c: Identify virt type of guest 2011-07-12 Daniel P. Berrange Add auditing of filesystems When passing through filesystems from the host to a guest, the host filesystem passed must be audited * src/conf/domain_audit.{c,h}: Add virDomainAuditFS 2011-07-12 Daniel P. Berrange Move qemu_audit.h helpers into shared code The LXC and UML drivers can both make use of auditing. Move the qemu_audit.{c,h} files to src/conf/domain_audit.{c,h} * src/conf/domain_audit.c: Rename from src/qemu/qemu_audit.c * src/conf/domain_audit.h: Rename from src/qemu/qemu_audit.h * src/Makefile.am: Remove qemu_audit.{c,h}, add domain_audit.{c,h} * src/qemu/qemu_audit.h, src/qemu/qemu_cgroup.c, src/qemu/qemu_command.c, src/qemu/qemu_driver.c, src/qemu/qemu_hotplug.c, src/qemu/qemu_migration.c, src/qemu/qemu_process.c: Update for changed audit API names 2011-07-12 Matthias Bolte esx: Initialize all instances of esxVMX_Data to zero To avoid using or freeing uninitialized memory or pointers. rpc: Fix compile error due to potentially unused parameter connectDBus is only used if HAVE_DBUS is set. Therefore mark it as potentially unused. 2011-07-12 Jim Fehlig Fix build when using polkit0 V2: Remove policy kit references from virNetServer and use DBus APIs directly, if available. 2011-07-12 Daniel P. Berrange Implement code to attach to external QEMU instances. Given a PID, the QEMU driver reads /proc/$PID/cmdline and /proc/$PID/environ to get the configuration. This is fed into the ARGV->XML convertor to build an XML configuration for the process. /proc/$PID/exe is resolved to identify the full command binary path After checking for name/uuid uniqueness, an attempt is made to connect to the monitor socket. If successful then 'info status' and 'info kvm' are issued to determine whether the CPUs are running and if KVM is enabled. * src/qemu/qemu_driver.c: Implement virDomainQemuAttach * src/qemu/qemu_process.h, src/qemu/qemu_process.c: Add qemuProcessAttach to connect to the monitor of an existing QEMU process 2011-07-12 Daniel P. Berrange Add monitor API for checking whether KVM is enabled When attaching to an external QEMU process, it is neccessary to check if the process is using KVM or not. This can be done using a monitor command * src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h, src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h, src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add API for checking if KVM is enabled 2011-07-12 Daniel P. Berrange Add a method for extracting QEMU argv from /proc To enable attaching to externally launched QEMU, we need to be able to reverse engineer a guest XML config based on the argv for a PID in /proc * src/qemu/qemu_command.c, src/qemu/qemu_command.h: Add qemuParseCommandLinePid which extracts QEMU config from argv in /proc, given a PID number 2011-07-12 Daniel P. Berrange Change extract pidfile & monitor config from QEMU command line When converting QEMU argv into a virDomainDefPtr, also extract the pidfile, monitor character device config and the monitor mode. * src/qemu/qemu_command.c, src/qemu/qemu_command.h: Extract pidfile & monitor config from QEMU argv * src/qemu/qemu_driver.c, tests/qemuargv2xmltest.c: Add extra params when calling qemuParseCommandLineString 2011-07-12 Daniel P. Berrange Keep pidfile path in qemuDomainObjPrivate struct Avoid re-formatting the pidfile path everytime we need it. Create it once when starting the guest, and preserve it until the guest is shutdown. * src/libvirt_private.syms, src/util/util.c, src/util/util.h: Add virFileReadPidPath * src/qemu/qemu_domain.h: Add pidfile field * src/qemu/qemu_process.c: Store pidfile path in qemuDomainObjPrivate 2011-07-12 Daniel P. Berrange Add a new 'virsh qemu-attach' command. This command allows libvirt to attach to an existing QEMU instance. $ qemu-kvm -cdrom ~/demo.iso \ -monitor unix:/tmp/demo,server,nowait \ -name foo \ -uuid cece4f9f-dff0-575d-0e8e-01fe380f12ea & $ QEMUPID=$! $ virsh qemu-attach $QEMUPID 2011-07-12 Daniel P. Berrange Define remote wire protocol & impls for virDomainQemuAttach This tweaks the RPC generator to cope with some naming conventions used for the QEMU specific APIs * daemon/remote.c: Server side dispatcher * src/remote/remote_driver.c: Client side dispatcher * src/remote/qemu_protocol.x: Wire protocol definition * src/rpc/gendispatch.pl: Use '$structprefix' in method names, fix QEMU flags and fix dispatcher method names 2011-07-12 Daniel P. Berrange Define a QEMU specific API to attach to a running QEMU process Introduce a new API in libvirt-qemu.so virDomainPtr virDomainQemuAttach(virConnectPtr domain, unsigned long long pid, unsigned int flags); This allows libvirtd to attach to an existing, externally launched QEMU process. This is useful for QEMU developers who prefer to launch QEMU themselves for debugging/devel reasons, but still want the benefit of libvirt based tools like virt-top, virt-viewer, etc * include/libvirt/libvirt-qemu.h: Define virDomainQemuAttach * src/driver.h, src/libvirt-qemu.c, src/libvirt_qemu.syms: Driver glue for virDomainQemuAttach 2011-07-12 Alex Jia qemu: update configuration file * src/qemu/qemu.conf: Add blkio controller into qemu.conf. 2011-07-12 Eric Blake qemu: avoid fd leak on core dump failure * src/qemu/qemu_driver.c (doCoreDump): Guarantee fd is closed. maint: rename virtaudit to match file contents * src/util/virtaudit.[ch]: Rename... * src/util/viraudit.[ch]: ...to match virAudit* API. * src/Makefile.am (UTIL_SOURCES): Reflect rename. * daemon/libvirtd.c: Likewise. * po/POTFILES.in: Likewise. * src/libvirt_private.syms: Likewise. * src/qemu/qemu_audit.c: Likewise. 2011-07-12 Oskari Saarenmaa remote/ssh: support for no_verify. Set StrictHostKeyChecking=no to auto-accept new ssh host keys if the no_verify extra parameter was specified. This won't disable host key checking for already known hosts. Includes a test and documentation. 2011-07-12 KAMEZAWA Hiroyuki Fix virsh inject-nmi man page * tools/virsh.pod: move the description which was misplaced 2011-07-12 Michal Privoznik domain_conf: Free temporary variable * src/conf/domain_conf.c: caller must free returned value of virXPathString in useserial 2011-07-12 Osier Yang esx: Fix a potential crash Initialize ptr data.datastorePathWithoutFileName as NULL, otherwise it might cause crash when trying to free it in cleanup. 2011-07-12 Eric Blake build: also check qemu_protocol for on-the-wire stability Since we are going to add some libvirt-qemu.so entry points in 0.9.4, we might as well start checking for RPC stability, just as for libvirt.so. * src/Makefile.am (PROTOCOL_STRUCTS): New variable. (remote_protocol-structs): Rename... (%_protocol-structs): ...and make more generic. * src/qemu_protocol-structs: New file. 2011-07-12 Eric Blake build: avoid requiring -lm log2() is heavy when ffs() can do the same thing. But ffs() requires gnulib support for mingw. This patch solves this linker error on Fedora 14. /usr/bin/ld: libvirt_lxc-domain_conf.o: undefined reference to symbol 'log2@@GLIBC_2.2.5' /usr/bin/ld: note: 'log2@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line /lib64/libm.so.6: could not read symbols: Invalid operation collect2: ld returned 1 exit status * .gnulib: Update to latest, for ffs. * bootstrap.conf (gnulib_modules): Import ffs. * src/conf/domain_conf.c (virDomainDefParseXML): Use ffs instead of log2. Reported by Dave Allan. 2011-07-11 Jim Fehlig Remove code no longer used after commit df0b57a9 2011-07-11 Matthias Bolte Add domain type checking The drivers were accepting domain configs without checking if those were actually meant for them. For example the LXC driver happily accepts configs with type QEMU. Add a check for the expected domain types to the virDomainDefParse* functions. 2011-07-11 Alex Jia remote: Fix memory leak Detected in valgrind run: ==9184== 1 bytes in 1 blocks are definitely lost in loss record 1 of 19 ==9184== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==9184== by 0x3073715F78: xdr_array (xdr_array.c:97) ==9184== by 0x4CF97C9: xdr_remote_domain_get_security_label_ret (remote_protocol.c:1696) ==9184== by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286) ==9184== by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318) ==9184== by 0x4CE3887: call (remote_driver.c:3933) ==9184== by 0x4CF71C6: remoteDomainGetSecurityLabel (remote_driver.c:1580) ==9184== by 0x4CCA480: virDomainGetSecurityLabel (libvirt.c:7340) ==9184== by 0x41993A: cmdDominfo (virsh.c:2414) ==9184== by 0x411E92: vshCommandRun (virsh.c:12730) ==9184== by 0x4211ED: main (virsh.c:14076) ==9184== ==9184== 2 bytes in 1 blocks are definitely lost in loss record 2 of 19 ==9184== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==9184== by 0x3073715F78: xdr_array (xdr_array.c:97) ==9184== by 0x4CF974F: xdr_remote_node_get_security_model_ret (remote_protocol.c:1713) ==9184== by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286) ==9184== by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318) ==9184== by 0x4CE3887: call (remote_driver.c:3933) ==9184== by 0x4CF6F96: remoteNodeGetSecurityModel (remote_driver.c:1648) ==9184== by 0x4CBF799: virNodeGetSecurityModel (libvirt.c:7382) ==9184== by 0x4197D7: cmdDominfo (virsh.c:2394) ==9184== by 0x411E92: vshCommandRun (virsh.c:12730) ==9184== by 0x4211ED: main (virsh.c:14076) ==9184== ==9184== 8 bytes in 1 blocks are definitely lost in loss record 3 of 19 ==9184== at 0x4A04A28: calloc (vg_replace_malloc.c:467) ==9184== by 0x3073715F78: xdr_array (xdr_array.c:97) ==9184== by 0x4CF9729: xdr_remote_node_get_security_model_ret (remote_protocol.c:1710) ==9184== by 0x4D08741: virNetMessageDecodePayload (virnetmessage.c:286) ==9184== by 0x4D00F78: virNetClientProgramCall (virnetclientprogram.c:318) ==9184== by 0x4CE3887: call (remote_driver.c:3933) ==9184== by 0x4CF6F96: remoteNodeGetSecurityModel (remote_driver.c:1648) ==9184== by 0x4CBF799: virNodeGetSecurityModel (libvirt.c:7382) ==9184== by 0x4197D7: cmdDominfo (virsh.c:2394) ==9184== by 0x411E92: vshCommandRun (virsh.c:12730) ==9184== by 0x4211ED: main (virsh.c:14076) ==9184== ==9184== LEAK SUMMARY: ==9184== definitely lost: 11 bytes in 3 blocks * src/remote/remote_driver.c: Avoid leak on remoteDomainGetSecurityLabel and remoteNodeGetSecurityModel. 2011-07-11 Eric Blake tests: simplify formatting The shell version would output 40 extra spaces for a test with a multiple of 40 sub-tests, and the C version can use the same printf optimization for avoiding a loop over single space output as the shell version. * tests/testutils.c (virtTestMain): Avoid loop for alignment. * tests/test-lib.sh: Fix formatting when counter is multiple of 40. 2011-07-11 Jim Fehlig Do not drop kernel cmdline for xen pv domains Kernel cmdline args can be passed to xen pv domains even when a bootloader is specified. The current config-to-sxpr mapping ignores cmdline when bootloader is present. Since the xend sub-driver is used with many xen toolstack versions, this patch takes conservative approach of adding an else block to existing !def->os.bootloader, and only appends sxpr if def->os.cmdline is non-NULL. V2: Fix existing testcase broken by this patch and add new testcases 2011-07-11 Wen Congyang kill vm if saving config failed in v3 protocol If virDomainSaveConfig() failed, we will return NULL to source, and the vm is still available to restart during confirm() step in v3 protocol. So we should kill it off in qemuMigrationFinish(). In v2 protocol, we should not set vm to NULL, because we hold a reference of vm and should unrefernce it. 2011-07-11 Wen Congyang RPC: fix argument's name fix typo error 2011-07-11 Michal Privoznik bios: Add support for SGA This patch creates new element which, at this time has only the attribute useserial='yes|no'. This attribute allow users to use Serial Graphics Adapter and see BIOS messages from the very first moment domain boots up. Therefore, users can choose boot medium, set PXE, etc. 2011-07-11 Daniel Veillard Fix rpm build with sanlock and without QEmu The qemu-sanlock.conf file is not installed in this case 2011-07-09 Matthias Bolte vbox: Fix logic in storage driver open function If the main driver is the vbox driver, then the open function has to return an error if the private data is invalid. 2011-07-09 Matthias Bolte tests: Improve output of tests that decide to skip at runtime Don't print OK/FAIL for tests that decide to be skipped after calling virtTestMain. Delay printing of the indentation before the first test until we know that the test didn't decide to be skipped. Also make the reconnect test use VIRT_TEST_MAIN. 2011-07-09 Matthias Bolte tests: Fix compressed test output padding logic The current logic tries to count from 1 to 40 and ignores paddings of 0 and 1 to 40. This doesn't work for counter + 1 mod 40 == 0 like here for counter value 159 TEST: virsh-all ........................................ 40 ........................................ 80 ........................................ 120 ....................................... 159 OK PASS: virsh-all Also seq isn't portable. Therefore, calculate the correct padding length directly and use printf to output it at once. 2011-07-09 Matthias Bolte tests: Use EXIT_AM_SKIP instead of 77 directly 2011-07-09 Matthias Bolte rpc: Fix whitespace problem in generated code Add missing line break and fix indention level. Reported by Cole Robinson. 2011-07-08 Federico Simoncelli sanlock: avoid lockspace setup when auto_disk_lease is off When auto_disk_lease is off we should avoid the automatic lockspace creation. 2011-07-08 Eric Blake libvirtd: diagnose explicitly requested but missing conf file Fixes test regression introduced in commit 8e2e4780. * daemon/libvirtd.c (daemonConfigLoad): Add argument. (main): Update caller. 2011-07-08 Michael Santos qemu: clean up OOM checks 2011-07-08 Daniel P. Berrange Don't exit if the libvirtd config does not exist It is common for the $HOME/.libvirt/libvirtd.conf file to not exist. Treat this situation as non-fatal since we can carry on with our default settings just fine. * daemon/libvirtd.c: Treat ENOENT as non-fatal when loading config 2011-07-08 Daniel P. Berrange Fix sending of reply to final RPC message The dispatch for the CLOSE RPC call was invoking the method virNetServerClientClose(). This caused the client connection to be immediately terminated. This meant the reply to the final RPC message was never sent. Prior to the RPC rewrite we merely flagged the connection for closing, and actually closed it when the next RPC call dispatch had completed. * daemon/remote.c: Flag connection for a delayed close * daemon/stream.c: Update to use new API for closing failed connection * src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h: Add support for a delayed connection close. Rename the virNetServerClientMarkClose method to virNetServerClientImmediateClose to clarify its semantics 2011-07-08 Daniel P. Berrange Fix leak of remote driver if final 'CLOSE' RPC call fails When closing a remote connection we issue a (fairly pointless) 'CLOSE' RPC call to the daemon. If this fails we skip all the cleanup of private data, but the virConnectPtr object still gets released as normal. This causes a memory leak. Since the CLOSE RPC call is pretty pointless, just carry on freeing the remote driver if it fails. * src/remote/remote_driver.c: Ignore failure to issue CLOSE RPC call 2011-07-08 Daniel P. Berrange Fix release of outgoing stream confirmation/abort message When sending back the final OK or ERROR message on completion of a stream, we were not decrementing the 'nrequests' tracker on the client. With the default requests limit of '5', this meant once a client had created 5 streams, they are unable to process any further RPC calls. There was also a bug when handling an error from decoding a message length header, which meant a client connection would not immediately be closed. * src/rpc/virnetserverclient.c: Fix release of request after stream completion & mark client for close on error 2011-07-08 Daniel P. Berrange Fix leak of 'msg' object in client stream code In one exit path we forgot to free the virNetMessage object causing a large memory leak for streams which send a lot of data. Some other paths were calling VIR_FREE directly instead of virNetMessageFree although this was (currently) harmless. * src/rpc/virnetclientstream.c: Fix leak of msg object * src/rpc/virnetclientprogram.c: Call virNetMessageFree instead of VIR_FREE 2011-07-08 Daniel P. Berrange Fix potential crash in libvirtd with active streams If a client disconnects while it has a stream active, there is a race condition which could see libvirtd crash. This is because the client struct may be freed before the last stream event has triggered. This is trivially solved by holding an extra reference on the client for the stream callbak * daemon/stream.c: Acquire reference on client when adding the stream callback 2011-07-08 Daniel P. Berrange Fix mistaken order of server cert/key parameters in constructor The virNetTLSContextNew was being passed key/cert parameters in the wrong order. This wasn't immediately visible because if virNetTLSContextNewPath was used, a second bug reversed the order of those parameters again. Only if the paths were manually specified in /etc/libvirt/libvirtd.conf did the bug appear * src/rpc/virnettlscontext.c: Fix order of params passed to virNetTLSContextNew 2011-07-08 Michal Privoznik graphics: add support for action_if_connected in qemu This option accepts 3 values: -keep, to keep current client connected (Spice+VNC) -disconnect, to disconnect client (Spice) -fail, to fail setting password if there is a client connected (Spice) 2011-07-08 Jiri Denemark util: Don't try to fchown files opened as non-root When virFileOpenAs is called with VIR_FILE_OPEN_AS_UID flag and uid/gid different from root/root while libvirtd is running as root, we fork a new child, change its effective UID/GID to uid/gid and run virFileOpenAsNoFork. It doesn't make any sense to fchown() the opened file in this case since we already know that uid/gid can access the file when open succeeds and one of the following situations may happen: - the file is already owned by uid/gid and we skip fchown even before this patch - the file is owned by uid but not gid because it was created in a directory with SETGID set, in which case it is desirable not to change the group - the file may be owned by a completely different user and/or group because it was created on a root-squashed or even all-squashed NFS filesystem, in which case fchown would most likely fail anyway 2011-07-08 Michal Privoznik qemu: Don't chown files on NFS share if dynamic_ownership is off When dynamic ownership is disabled we don't want to chown any files, not just local. 2011-07-08 John Williams microblaze: Add architecture support Add libvirt support for MicroBlaze architecture as a QEMU target. Based on mips/mipsel pattern. 2011-07-08 Jim Fehlig Skip some xen tests if xend is not running Currently, the xen statstest and reconnect tests are only compiled if xend is running. Compile them unconditionally if xen headers are present, but skip the tests at runtime if xend is not running. This is in response to Eric's suggestion here https://www.redhat.com/archives/libvir-list/2011-July/msg00367.html 2011-07-07 Eric Blake util: drop unused safezero argument No caller was using the flags argument, and this function is internal only, so we might as well skip it. * src/util/util.h (safezero): Update signature. * src/util/util.c (safezero): Update function. * src/locking/lock_driver_sanlock.c (virLockManagerSanlockSetupLockspace) (virLockManagerSanlockCreateLease): Update all callers. * src/storage/storage_backend.c (createRawFile): Likewise. 2011-07-07 Eric Blake conf: prefer unsigned int for flags * src/conf/domain_conf.c (virDomainGraphicsDefParseXML) (virDomainDeviceInfoFormat, virDomainDeviceInfoParseXML) (virDomainDiskDefParseXML, virDomainControllerDefParseXML) (virDomainFSDefParseXML, virDomainNetDefParseXML) (virDomainChrDefParseTargetXML, virDomainChrDefParseXML) (virDomainSmartcardDefParseXML, virDomainInputDefParseXML) (virDomainTimerDefParseXML, virDomainSoundDefParseXML) (virDomainWatchdogDefParseXML, virDomainMemballoonDefParseXML) (virDomainVideoDefParseXML) (virDomainHostdevSubsysUsbDefParseXML) (virDomainHostdevSubsysPciDefParseXML) (virDomainHostdevDefParseXML, virSecurityLabelDefParseXML) (virDomainVcpuPinDefParseXML, virDomainDefParseXML) (virDomainDefParse, virDomainDiskDefFormat) (virDomainControllerDefFormat, virDomainFSDefFormat) (virDomainNetDefFormat, virDomainChrSourceDefFormat) (virDomainChrDefFormat, virDomainSmartcardDefFormat) (virDomainSoundDefFormat, virDomainMemballoonDefFormat) (virDomainWatchdogDefFormat, virDomainVideoDefFormat) (virDomainInputDefFormat, virDomainGraphicsDefFormat) (virDomainHostdevDefFormat, virDomainObjFormat): Switch signature. (virDomainObjTaint, virDomainSaveStatus): Use unsigned flags. 2011-07-07 Eric Blake drivers: prefer unsigned int for flags Now that the public APIs always use unsigned flags, the internal driver callbacks might as well do likewise. * src/driver.h (vrDrvOpen, virDrvDomainCoreDump) (virDrvDomainGetXMLDesc, virDrvNetworkGetXMLDesc) (virDrvNWFilterGetXMLDesc): Update type. * src/remote/remote_protocol.x (remote_open_args) (remote_domain_core_dump_args, remote_domain_get_xml_desc_args) (remote_network_get_xml_desc_args) (remote_nwfilter_get_xml_desc_args): Likewise. * src/test/test_driver.c: Update clients. * src/remote/remote_driver.c: Likewise. * src/xen/xen_hypervisor.c: Likewise. * src/xen/xen_hypervisor.h: Likewise. * src/xen/xen_driver.c: Likewise. * src/xen/xend_internal.c: Likewise. * src/xen/xend_internal.h: Likewise. * src/xen/xm_internal.c: Likewise. * src/xen/xm_internal.h: Likewise. * src/xen/xs_internal.c: Likewise. * src/xen/xs_internal.h: Likewise. * src/xen/xen_inotify.c: Likewise. * src/xen/xen_inotify.h: Likewise. * src/phyp/phyp_driver.c: Likewise. * src/openvz/openvz_driver.c: Likewise. * src/vmware/vmware_driver.c: Likewise. * src/vbox/vbox_driver.c: Likewise. * src/vbox/vbox_tmpl.c: Likewise. * src/xenapi/xenapi_driver.c: Likewise. * src/esx/esx_driver.c: Likewise. * src/esx/esx_interface_driver.c: Likewise. * src/esx/esx_network_driver.c: Likewise. * src/esx/esx_storage_driver.c: Likewise. * src/esx/esx_device_monitor.c: Likewise. * src/esx/esx_secret_driver.c: Likewise. * src/esx/esx_nwfilter_driver.c: Likewise. * src/interface/netcf_driver.c: Likewise. * src/nwfilter/nwfilter_driver.c: Likewise. * src/libxl/libxl_driver.c: Likewise. * src/qemu/qemu_driver.c: Likewise. * src/lxc/lxc_driver.c: Likewise. * src/uml/uml_driver.c: Likewise. * src/network/bridge_driver.c: Likewise. * src/secret/secret_driver.c: Likewise. * src/storage/storage_driver.c: Likewise. * src/node_device/node_device_hal.c: Likewise. * src/node_device/node_device_udev.c: Likewise. * src/remote_protocol-structs: Likewise. 2011-07-07 Eric Blake public API: prefer unsigned int for flags Most APIs use 'unsigned int flags'; but a few stragglers were using a signed value. In particular, the vir*GetXMLDesc APIs were split-brain, with inconsistent choice of types. Although it is an API break to use 'int' instead of 'unsigned int', it is ABI compatible (pre-compiled apps will have no difference in behavior), and generally apps can be recompiled without any issue (only rare apps that compiled with extremely high warning levels, or which pass libvirt API around as typed function pointers, would have to make any code changes to deal with the change). The migrate APIs use 'unsigned long flags', which can't be changed, due to ABI constraints. This patch intentionally touches only the public API, to prove the claim that most existing code (including driver callbacks and virsh) still compiles just fine in spite of the type change. * include/libvirt/libvirt.h.in (virConnectOpenAuth) (virDomainCoreDump, virDomainGetXMLDesc, virNetworkGetXMLDesc) (virNWFilterGetXMLDesc): Use unsigned int for flags. (virDomainHasCurrentSnapshot): Use consistent spelling. * src/libvirt.c (virConnectOpenAuth, virDomainCoreDump) (virDomainGetXMLDesc, virNetworkGetXMLDesc) (virNWFilterGetXMLDesc, do_open): Update accordingly. 2011-07-07 Eric Blake maint: print flags in hex during debug Debugging decimal flags is a pain. * src/libvirt.c: Always print flags in hex. 2011-07-07 Eric Blake virsh: make destroy sound less scary Destroy has a rather negative English connotation. Try to reduce the impact, so newbies aren't as scared to use it. * tools/virsh.c: Tweak all destroy documentation. * tools/virsh.pod: Likewise. 2011-07-07 Eric Blake build: use gnulib pthread_sigmask Gnulib finally learned how to do pthread_sigmask on mingw. * .gnulib: Update to latest, for pthread_sigmask. * bootstrap.conf (gnulib_modules): Add pthread_sigmask. * configure.ac (AC_CHECK_FUNCS): Drop redundant check. * src/rpc/virnetclient.c (virNetClientSetTLSSession) (virNetClientIOEventLoop): Make code unconditional. * src/util/command.c (virFork): Likewise. * tools/virsh.c (doMigrate, cmdMigrate): Likewise. 2011-07-07 Eric Blake build: fix virBufferVasprintf on mingw Gnulib documents that mingw vsnprintf is broken (it returns -1 on out-of-space, instead of the count of what would have been printed); but while we were using the snprintf wrapper, we had not yet been using the vsnprintf wrapper. Meanwhile, mingw (but not mingw64) has a replacement snprintf that fixes return values, but still lacks %1$s support; so in that case, gnulib didn't replace snprintf, but libintl then went ahead and installed a version that supported %1$s but not return values. Gnulib has since been fixed to guarantee that the snprintf module will always guarantee the constraints needed by libintl. Also, we want to guarantee that strdup sets errno on failure. * .gnulib: Update to latest, for vsnprintf fix. * bootstrap.conf (gnulib_modules): Add vsnprintf, strdup-posix. Reported by Matthias Bolte. 2011-07-07 Jim Fehlig Fix compilation of statstest.c during make check 2011-07-07 Matthias Bolte python: Fix bogus label placement 2011-07-07 Laine Stump util: close the ioctl socket at the end of if(Get|Set)MacAddress Otherwise this will leak an fd each time one of these functions is called. util: rename err_exit to cleanup in interface.c This brings it in line with the recommendations in HACKING. 2011-07-06 Minoru Usui sysinfo: delete unnecessary white space of sysinfo. * Trim each element and delete null entry of sysinfo by virSkipSpacesBackwards(). 2011-07-06 Eric Blake util: add virTrimSpaces The next patch wants to adjust an end pointer to trim trailing spaces but without modifying the underlying string, but a more generally useful ability to trim trailing spaces in place is also worth providing. * src/util/util.h (virTrimSpaces, virSkipSpacesBackwards): New prototypes. * src/util/util.c (virTrimSpaces, virSkipSpacesBackwards): New functions. * src/libvirt_private.syms (util.h): Export new functions. Inspired by a patch by Minoru Usui. 2011-07-06 Eric Blake util: fix virSkipSpaces Most clients of virSkipSpaces don't want to omit backslashes. Also, open-coding the list of spaces is not as nice as using c_isspace. * src/util/util.c (virSkipSpaces): Use c_isspace. (virSkipSpacesAndBackslash): New function. * src/util/util.h (virSkipSpacesAndBackslash): New prototype. * src/xen/xend_internal.c (sexpr_to_xend_topology): Update caller. * src/libvirt_private.syms (util.h): Export new function. 2011-07-06 Eric Blake docs: minor whitespace cleanups No change in wording. One spacing change in a
, noticed because
	of odd XML formatting online; the rest is in free-flowing text to
	make it easier to see nesting levels in the document.

	* docs/formatdomain.html.in: Adjust spacing.  Break long lines.

2011-07-06  Matthias Bolte  

	Reduce code duplication in virFileMakePath(Helper)
	Move stat and mkdir to virFileMakePathHelper.

	Also use the stat result to detect whether the existing path
	is a directory and set errno accordingly if it's not.

2011-07-06  Matthias Bolte  

	apparmor: Finish incomplete renaming of relabel to norelabel
	Commit 693eac388f1759d was incomplete here.

2011-07-06  Daniel P. Berrange  

	Fix default value of security label 'relabel' attribute
	When no  is present in the XML, the virDomainSeclabelDef
	struct is left as all zeros. Unfortunately, this means it gets setup
	as type=dynamic, with relabel=no, which is an illegal combination.

	Change the 'bool relabel' attribute in virDomainSeclabelDef to
	the inverse 'bool norelabel' so that the default initialization
	is sensible

	* src/conf/domain_conf.c, src/conf/domain_conf.h,
	  src/security/security_apparmor.c, src/security/security_selinux.c:
	  Replace 'relabel' with 'norelabel'

2011-07-06  Matthias Bolte  

	Fix return value semantic of virFileMakePath
	Some callers expected virFileMakePath to set errno, some expected
	it to return an errno value. Unify this to return 0 on success and
	-1 on error. Set errno to report detailed error information.

	Also optimize virFileMakePath if stat fails with an errno different
	from ENOENT.

2011-07-06  Matthias Bolte  

	nodedev: Let check_fc_host_linux report errors to the caller

2011-07-06  Jean-Baptiste Rouault  

	Fix compilation error when SASL support is disabled
	This patch adds #if HAVE_SASL where needed in libvirtd.h

2011-07-05  Guannan Ren  

	pci: initialize state values on reattach
	add a new API pciDeviceReAttachInit() in pci.c to initialize state values for nodedev reattach

	Initialize three state value of device driver to 1. This is just for a new call to
	qemudNodeDeviceReAttach()

2011-07-05  Laine Stump  

	xml: create an RNG file for common types and move some definitions there
	domain.rng, network.rng, and interface.rng already use a few of the
	same types (or in some cases *should* but don't), and an upcoming code
	change will have them sharing even more. To prepare for that, this
	patch takes those common data type definitions and moves them into
	basictypes.rng.

	This may break some rule about the need to RNG files to be autonomous
	or something, but I saw that storageencryption.rng is used in this
	way, so I figured it must not be completely against the law...

2011-07-05  Laine Stump  

	docs: sort the file list in the schemas Makefile

	qemu: check flags in qemuDomainGetXMLDesc
	Although most functions with flags check to verify no application is
	passing in flag bits that are currently undefined, for some reason
	this function wasn't.

2011-07-05  Laine Stump  

	conf: fix domain parse flags
	* Change all flags args from int to unsigned int

	* Allow passing flags in virDomainObjParseFile (and propogate those
	  flags all the way down the call chain). Previously the flags were
	  hardcoded (to VIR_DOMAIN_XML_INTERNAL_STATUS) several layers down
	  the chain. Pass that value in at the one place that is currently
	  calling virDomainObjParseFile.

2011-07-05  Matthias Bolte  

	tests: Disable networkxml2argvtest when configured without network
	Reported by Ruben Kerkhof.

	qemu: Fix virFileMakePath error handling in snapshot creation
	virFileMakePath returns an errno value on error, that will never
	be negative. An virFileMakePath error would have been ignored here,
	instead of being reported correctly.

2011-07-05  Matthias Bolte  

	conf: Fix declaration of virNetworkDNS(Hosts)Def struct
	The struct A {} A; construct triggers a linker error on OSX about
	duplicate symbols. This also differs from the common struct style.

	Switch to common style to fix this.

	Reported by Justin Clift.

2011-07-04  Daniel P. Berrange  

	Add documentation for the seclabel XML element
	The domain XML documentation is missing information about the
	 element used by security drivers

	* formatdomain.html.in: Document 

2011-07-04  Daniel P. Berrange  

	Allow for resource relabelling with static labels
	Add a new attribute to the  XML to allow resource
	relabelling to be enabled with static label usage.

	  
	    
	  

	* docs/schemas/domain.rng: Add relabel attribute
	* src/conf/domain_conf.c, src/conf/domain_conf.h: Parse
	  the 'relabel' attribute
	* src/qemu/qemu_process.c: Unconditionally clear out the
	  'imagelabel' attribute
	* src/security/security_apparmor.c: Skip based on 'relabel'
	  attribute instead of label type
	* src/security/security_selinux.c: Skip based on 'relabel'
	  attribute instead of label type and fill in 
	  attribute if relabel is enabled.

2011-07-04  Daniel P. Berrange  

	Allow a base label to be specified in dynamic labelling mode
	Normally the dynamic labelling mode will always use a base
	label of 'svirt_t' for VMs. Introduce a  field
	in the  XML to allow this base label to be changed

	eg

	   
	     system_u:object_r:virt_t:s0
	   

	* docs/schemas/domain.rng: Add 
	* src/conf/domain_conf.c, src/conf/domain_conf.h: Parsing
	  of base label
	* src/qemu/qemu_process.c: Don't reset 'model' attribute if
	  a base label is specified
	* src/security/security_apparmor.c: Refuse to support base label
	* src/security/security_selinux.c: Use 'baselabel' when generating
	  label, if available

2011-07-04  Osier Yang  

	storage: Do not override the exact error of createRawFile
	virStorageBackendCreateRaw: createRawFile already reported the
	exact error.

	Before the fix:

	error: Failed to create vol vol-create.img
	error: cannot create path '/var/lib/libvirt/images/vol-create.img': Unknown error 18446744073709551597

	After the fix:

	error: Failed to create vol vol-create.img
	error: cannot fill file '/var/lib/libvirt/images/vol-create.img': No space left on device

2011-07-04  Daniel Veillard  

	Release of libvirt-0.9.3
	* configure.ac docs/news.html.in libvirt.spec.in: update for the
	  new release
	* po/*.po*: updated and regenerated localizations

2011-07-04  Wen Congyang  

	initialize pointer to NULL
	cmd is not initialized to NULL, but we try to freed it if we
	meet some error.

2011-07-04  Eric Blake  

	storage: avoid crash on parse error
	Coverity detected that we could crash on bogus input.  Meanwhile,
	strtok_r is rather heavy compared to strchr.

	* src/storage/storage_backend_iscsi.c (virStorageBackendIQNFound):
	  Check for parse failure, and use lighter-weight functions.

2011-07-04  Eric Blake  

	vmware: avoid null deref on failed lookup
	* src/vmware/vmware_driver.c (vmwareDomainReboot): Check error
	  before dereferencing memory.

2011-07-04  Eric Blake  

	qemu: avoid null deref on low memory
	Detected by Coverity.  qemuDomainEventQueue requires a non-NULL
	pointer; most callers silently drop the event if we encountered
	and OOM situation trying to create the event.

	* src/qemu/qemu_migration.c (qemuMigrationFinish): Check for OOM.

2011-07-04  Eric Blake  

	cgroup: silence coverity warning
	Coverity noted that most clients reacted to failure to hash; but in
	a best-effort kill loop, we can ignore failure.

	* src/util/cgroup.c (virCgroupKillInternal): Ignore hash failure.

2011-07-04  Eric Blake  

	rpc: silence coverity warning
	Coverity noted that 4 out of 5 calls to virNetClientStreamRaiseError
	checked the return value.  This case expects a particular value, so
	warn if our expectations went wrong due to some bug elsewhere.

	* src/rpc/virnetclient.c (virNetClientCallDispatchStream): Warn on
	  unexpected scenario.

2011-07-04  Eric Blake  

	qemu: silence coverity warnings
	Coverity warns if the majority of callers check a function for
	errors, but a few don't; but in qemu_audit and qemu_domain, the
	choice to not check for failures was safe.  In qemu_command, the
	failure to generate a uuid can only occur on a bad pointer.

	* src/qemu/qemu_audit.c (qemuAuditCgroup): Ignore failure to get
	  cgroup controller.
	* src/qemu/qemu_domain.c (qemuDomainObjEnterMonitor)
	  (qemuDomainObjEnterMonitorWithDriver): Ignore failure to get
	  timestamp.
	* src/qemu/qemu_command.c (qemuParseCommandLine): Check for error.

2011-07-04  Eric Blake  

	rpc: avoid memory leak on error
	Detected by Coverity.  The leak is on an error path, but I'm not
	sure whether that path is likely to be triggered in practice.

	* src/rpc/virnetserverservice.c (virNetServerServiceAccept): Plug leak.

2011-07-04  Eric Blake  

	rpc: fix logic bug
	Spotted by Coverity.  If we don't update tmp each time through
	the loop, then if the filter being removed was not the head of
	the list, we accidentally lose all filters prior to the one we
	wanted to remove.

	* src/rpc/virnetserverclient.c (virNetServerClientRemoveFilter):
	    Don't lose unrelated filters.

2011-07-02  Eric Blake  

	pci: avoid memory leak on error
	Detected by Coverity.  Some, but not all, error paths were clean;
	but they were repetitive so I refactored them.

	* src/util/pci.c (pciGetDevice): Plug leak.

2011-07-02  Eric Blake  

	interface: avoid memory leak on parse error
	Detected by Coverity.  Unlikely to hit unless the file contents
	were corrupted.

	* src/util/interface.c (ifaceRestoreMacAddress): Plug leak.

2011-07-02  Eric Blake  

	libvirtd: avoid memory leak on OOM
	Detected by Coverity; only strikes on OOM so not serious.

	* daemon/libvirtd.c (daemonPidFilePath): Plug leak.

2011-07-02  Eric Blake  

	build: avoid 'make syntax-check' failure
	* tests/utiltest.c (DO_TEST): Fix indentation for cppi.

2011-07-01  Matthias Bolte  

	tests: Add a general util test
	Move non-esx specific tests from esxutilstest there and add a
	test for virParseVersionString.

2011-07-01  Eric Blake  

	util: choose whether to require micro in version strings
	To avoid regressions, we let callers specify whether to require a
	minor and micro version.  Callers that were parsing uname() output
	benefit from defaulting to 0, whereas callers that were parsing
	version strings from other sources should not change in behavior.

	* src/util/util.c (virParseVersionString): Allow caller to choose
	whether to fail if minor or micro is missing.
	* src/util/util.h (virParseVersionString): Update signature.
	* src/esx/esx_driver.c (esxGetVersion): Update callers.
	* src/lxc/lxc_driver.c (lxcVersion): Likewise.
	* src/openvz/openvz_conf.c (openvzExtractVersionInfo): Likewise.
	* src/uml/uml_driver.c (umlGetVersion): Likewise.
	* src/vbox/vbox_MSCOMGlue.c (vboxLookupVersionInRegistry):
	Likewise.
	* src/vbox/vbox_tmpl.c (vboxExtractVersion): Likewise.
	* src/vmware/vmware_conf.c (vmwareExtractVersion): Likewise.
	* src/xenapi/xenapi_driver.c (xenapiGetVersion): Likewise.
	Reported by Matthias Bolte.

2011-07-01  Eric Blake  

	build: consistently use CFLAGS
	According to the automake manual, CPPFLAGS (aka INCLUDES, as spelled
	in automake 1.9.6) should only include -I, -D, and -U directives; more
	generic directives like -Wall belong in CFLAGS since they affect more
	phases of the build process.  Therefore, we should be sticking CFLAGS
	additions into a CFLAGS container, not a CPPFLAGS container.

	* src/Makefile.am (libvirt_driver_vmware_la_CFLAGS): Use AM_CFLAGS.
	(INCLUDES): Move CFLAGS items...
	(AM_CFLAGS): ...to their proper location.
	* python/Makefile.am (INCLUDES, AM_CFLAGS): Likewise.
	* tests/Makefile.am (INCLUDES, AM_CFLAGS): Likewise.
	(commandtest_CFLAGS, commandhelper_CFLAGS)
	(virnetmessagetest_CFLAGS, virnetsockettest_CFLAGS): Use AM_CFLAGS.

2011-07-01  Daniel Veillard  

	Explicitely invoke python for API doc generator
	This fixes the problem of not finding python in /usr/bin
	which broke build on FreeBSD

2011-07-01  Scott Moser  

	fix virParseVersionString with linux 3.0
	linux 3.0 has no micro version number, and that is causing problems
	for virParseVersionString.  The patch below should allow for:
	  major
	  major.minor
	  major.minor.micro

	If major or minor are not present they just default to zero.
	We found this in Ubuntu (https://bugs.launchpad.net/bugs/802977)

2011-07-01  Eric Blake  

	build: remove dead variables
	Detected by Coverity.  No real harm in leaving these, but fixing
	them cuts down on the noise for future analysis.

	* src/rpc/virnetserver.c (virNetServerAddService): Delete unused
	entry.
	* src/util/sysinfo.c (virSysinfoRead): Delete dead assignment to
	base.

2011-07-01  Daniel P. Berrange  

	Remove bogus jsondata.h file reference which does not exist
	* tests/Makefile.am: Remove jsondata.h

2011-07-01  Eric Blake  

	build: simplify sanlock distribution
	EXTRA_DIST files should unconditionally be part of the tarball,
	rather than depending on the presence of sanlock-devel.

	Meanwhile, parallel builds could fail if we don't use mkdir -p.

	* src/Makefile.am (EXTRA_DIST): Always ship sanlock .aug and
	template .conf files.
	(%-sanlock.conf): Use MKDIR_P.

2011-07-01  Eric Blake  

	build: allow 'make syntax-check' on fresh checkout
	For good or for bad, I did a fresh checkout, ./autogen.sh, then
	'configure', then 'make syntax-check', and was surprised that it
	failed.  Running 'make' before 'make syntax-check' cleaned up the
	issue, but this patch makes it work up front.

	* cfg.mk (sc_po_check): Add prerequisites.

2011-06-30  Eric Blake  

	build: ignore generated file
	* .gitignore: Exempt jsontest binary.

2011-06-30  Daniel P. Berrange  

	Add conditionals to allow build without SASL
	* daemon/libvirtd.c, daemon/remote.c: Add #if HAVE_SASL and
	  suitable function stubs to allow build without SASL

2011-06-30  Eric Blake  

	build: avoid double-close bug with pipe2
	Based on Coverity's finding on the previous patch, I audited
	gnulib's pipe2 code and found that we had the potential for
	a subtle double-close bug, unless gnulib guarantees that the
	contents of the fd array are unchanged on pipe2() failure.

	* .gnulib: Update to latest, for pipe2 fix.

2011-06-30  Eric Blake  

	rpc: avoid freeing uninitialized variable
	Detected by Coverity.  Both are instances of bad things happening
	if pipe2 fails; the virNetClientNew failure could free garbage,
	and virNetSocketNewConnectCommand could close random fds.

	Note: POSIX doesn't guarantee the contents of fd[0] and fd[1]
	after pipe failure: http://austingroupbugs.net/view.php?id=467
	We may need to introduce a virPipe2 wrapper that guarantees
	that on pipe failure, the fds are explicitly set to -1, rather
	than our current state of assuming the fds are unchanged from
	their value prior to the failed pipe call.

	* src/rpc/virnetclient.c (virNetClientNew): Initialize variable.
	* src/rpc/virnetsocket.c (virNetSocketNewConnectCommand):
	Likewise.

2011-06-30  Eric Blake  

	virsh: avoid uninitialized variable
	Detected by Coverity; neither vshCmddefHelp nor vshCmdOptParse
	was initializing opts_required.

	* tools/virsh.c (vshCmddefOptParse): Always initialize bitmaps.

2011-06-30  Eric Blake  

	virsh: avoid integer overflow
	Detected by Coverity.  info.nrVirtCpu is unsigned short, but if
	cpumaplen is int, then the product of the two in vshMalloc risks
	unintended sign extension.  cmdVcpuinfo had already solved this
	by using size_t cpumaplen.

	* tools/virsh.c (cmdVcpuPin): Use correct type.

2011-06-30  Daniel P. Berrange  

	Fix stream procedure number for virDomainMigratePrepareTunnel3
	The virDomainMigratePrepareTunnel3 impl in the remote driver
	was using the procedure number for the virDomainMigratePrepareTunnel
	method. This doesn't work out so well, because it makes the server
	ignore & drop all stream packets

	* src/remote/remote_driver.c: Fix procedure for PrepareTunnel3

2011-06-30  Daniel P. Berrange  

	Send back an error if we get unexpected stream control message
	We ignore any stream data packets which come in for streams which
	are not registered, since these packets are async and do not have
	a reply. If we get a stream control packet though we must send back
	an actual error, otherwise a (broken) client may hang forever
	making it hard to diagnose the client bug.

	* src/rpc/virnetserverprogram.c: Send back error for unexpected
	  stream control messages

2011-06-30  Daniel P. Berrange  

	Fix release of virNetMessagePtr instances in streams processing
	If a message packet for a invalid stream is received it is just
	free'd. This is not good because it doesn't let the client RPC
	request counter decrement. If a stream is shutdown with pending
	packets the message also isn't released properly because of an
	incorrect header type

	* daemon/stream.c: Fix message header type
	* src/rpc/virnetserverprogram.c: Send dummy reply instead of
	  free'ing ignored stream message

2011-06-30  Daniel P. Berrange  

	Add missing include of signal.h in virnetsocket.c
	virNetSocketFree uses kill(SIGTERM) so we must include
	signal.h for the definitions

	* src/rpc/virnetsocket.c: Include signal.h

2011-06-30  Daniel P. Berrange  

	Add test case for parsing JSON docs
	While investigating some memory leaks it was unclear whether the
	JSON code correctly free'd all memory during parsing. Add a test
	case which can be run under valgrind to clearly demonstrate that
	the parser is leak free.

	* tests/Makefile.am: Add 'jsontest'
	* tests/jsontest.c: A few simple JSON parsing tests

2011-06-30  Daniel P. Berrange  

	Fix potential crash when saving guests
	The qemudDomainSaveFlag method will call EndJob on the 'vm'
	object it is passed in. This can result in the 'vm' object
	being free'd if the last reference is removed. Thus no caller
	of 'qemudDomainSaveFlag' must *ever* reference 'vm' again
	upon return.

	Unfortunately qemudDomainSave and qemuDomainManagedSave
	both call 'virDomainObjUnlock', which can result in a
	crash. This is non-deterministic since it involves a race
	with the monitor I/O thread.

	Fix this by making qemudDomainSaveFlag responsible for
	calling virDomainObjUnlock instead.

	* src/qemu/qemu_driver.c: Fix potential use after free
	  when saving guests

2011-06-30  Daniel P. Berrange  

	Fix uninitialized value in QEMU monitor FD sending code
	The 'char control[CMSG_SPACE(sizeof(int))];' was not being
	wiped, so could potentially contain uninitialized bytes.
	While this was harmless in this case, it caused complaints
	from valgrind

	* src/qemu/qemu_monitor.c: memset 'control' variable
	  in qemuMonitorIOWriteWithFD

2011-06-30  Daniel P. Berrange  

	Fix leak of JSON object for events
	The event handler functions do not free the virJSONValuePtr
	object. Every event received from a VM thus caused a memory
	leak

	* src/qemu/qemu_monitor_json.c: Fix leak of event object

2011-06-30  Daniel P. Berrange  

	Remove bogus warning message in JSON code
	* src/util/json.c: Remove warning message

2011-06-30  Daniel P. Berrange  

	Fix use of uninitialized memory when releasing PCI slots
	The 'function' field in the PCI address was not correctly
	initialized, so it was building the wrong address address
	string and so not removing all functions from the in use
	list.

	* src/qemu/qemu_command.c: Fix initialization of PCI function

2011-06-30  Daniel P. Berrange  

	Fix leak of virStreamPtr object with callback added in fdstream impl
	When adding a callback to an FD stream, we take an extra reference
	on the virStreamPtr instance. We forgot to registered a free function
	with the callback, so when the callback was removed, the extra
	reference held on virStreamPtr was not released.

	* src/fdstream.c: Use a free callback to release reference on
	  virStreamPtr when removing callback

2011-06-30  Daniel P. Berrange  

	Fix leak of mdnsGroupName in virNetServer object
	* src/rpc/virnetserver.c: Free mdnsGroupName

2011-06-30  Daniel P. Berrange  

	Fix release of filtered stream messages
	The stream code was reusing a stream message object before
	it was removed from the linked list of filtered messages.
	This caused any later queued messages to be completely lost.

	* daemon/stream.c: Delay reuse of stream message until
	  after it is removed from the queue

2011-06-30  Daniel P. Berrange  

	Ensure RPC message is cleared before being reused
	To save on memory reallocation, virNetMessage instances that
	have been transmitted, may be reused for a subsequent incoming
	message. We forgot to clear out the old data of the message
	fully, which caused later confusion upon read.

	* src/rpc/virnetserverclient.c: memset entire message before
	  reusing it

2011-06-30  Daniel P. Berrange  

	Fix hardcoded limit on client requests in RPC code
	The virNetServerClient object had a hardcoded limit of 10 requests
	per client. Extend constructor to allow it to be passed in as a
	configurable variable. Wire this up to the 'max_client_requests'
	config parameter in libvirtd

	* daemon/libvirtd.c: Pass max_client_requests into services
	* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Pass
	  nrequests_client_max to clients
	* src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h: Allow
	  configurable request limit

2011-06-30  Osier Yang  

	tests: Add valgrind.supp into EXTRA_DIST

	virsh: Fix a problem of buildPoolXML
	It doesn't generate "" and "" nodes for ""
	even if they are explicitly specified. This patch fixes it.

2011-06-30  Wen Congyang  

	lock qemu_driver early in qemuGetSchedulerParametersFlags()
	If we pass VIR_DOMAIN_AFFECT_LIVE | VIR_DOMAIN_AFFECT_CONFIG to
	qemuGetSchedulerParametersFlags() or *nparams is less than 1,
	we will unlock qemu_driver without locking it. It's very dangerous.

	We should lock qemu_driver after calling virCheckFlags().

2011-06-30  Wen Congyang  

	save domain status after modifing vcpupin
	We should save domain status after modifing vcpupin. If not,
	we will get wrong vcpupin information after rebooting libvirtd.

2011-06-30  Wen Congyang  

	Fix memory leak in virDomainVcpuPinDel()
	virDomainVcpuPinDefFree() does not free def->cputune.vcpupin if nvcpupin
	is 0, and does not set def->cputune.vcpupin to NULL.

	If we set nvcpupin to 0 but do not free vcpupin, vcpupin will not be freed
	when vm->def is freed.

	Use VIR_FREE() instead of virDomainVcpuPinDefFree() to free the memory
	and set def->cputune.vcpupint to NULL.

2011-06-29  Eric Blake  

	build: avoid pod2man on tarball
	virt-sanlock-cleanup.8 has static contents (no dependency on
	configure), but is generated by pod2man (a perl dependency that
	maintainers must have, but which ordinary tarball users need
	not have).  Therefore, ensure that it is always part of the
	tarball, even though it is only conditionally installed.

	This is similar to commit 6db98a2d4b, but made simpler by the fact
	that the .8 page is static content.

	* tools/Makefile.am (EXTRA_DIST): Add virt-sanlock-cleanup.8.

2011-06-29  Minoru Usui  

	sysinfo: fix illegal NULL return
	If virSysinfoParse{BIOS,System,Processor,Memory}()
	can't find newline('\n'), these return NULL.
	This patch fixes this.

2011-06-29  Daniel P. Berrange  

	Ensure that EOF is dispatched to the stream callback
	When the remote client receives end of file on the stream
	it never invokes the stream callback. Applications relying
	on async event driven I/O will thus never see the EOF
	condition on the stream

	* src/rpc/virnetclient.c, src/rpc/virnetclientstream.c:
	  Ensure EOF is dispatched

2011-06-29  Daniel P. Berrange  

	Fix locking wrt virNetClientStreamPtr object
	The client stream object can be used independently of the
	virNetClientPtr object, so must have full locking of its
	own and not rely on any caller.

	* src/remote/remote_driver.c: Remove locking around stream
	  callback
	* src/rpc/virnetclientstream.c: Add locking to all APIs
	  and callbacks

2011-06-29  Daniel P. Berrange  

	Avoid referencing NULL pointer when copying stream error
	* src/rpc/virnetclientstream.c: Avoid referencing NULL

2011-06-29  Daniel P. Berrange  

	Avoid free'ing a filtered RPC message in the server
	When a filter steals an RPC message, that message must
	not be freed, except by the filter code itself

	* src/rpc/virnetserverclient.c: Don't free stolen RPC
	  messages

2011-06-29  Daniel P. Berrange  

	Improve two log messages in virNetMessage
	Improve log messages issued when encountering a bogus
	message length to include the actual length and the
	limit violated

	* src/rpc/virnetmessage.c: Improve log messages

2011-06-29  Daniel P. Berrange  

	Ensure empty payload is written upon stream completion
	On stream completion it is neccessary to send back a
	message with an empty payload. The message header was
	not being filled out correctly, since we were not writing
	any payload. Add a method for encoding an empty payload
	which updates the message headers correctly.

	* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Add
	  a virNetMessageEncodePayloadEmpty method
	* src/rpc/virnetserverprogram.c: Write empty payload on
	  stream completion

2011-06-29  Daniel P. Berrange  

	Lower logging level when failing to register socket watch
	The RPC client treats failure to register a socket watch
	as non-fatal, since we do not mandate that a libvirt client
	application provide an event loop implementation. It is
	thus inappropriate to a log a message at VIR_LOG_WARN

	* src/rpc/virnetsocket.c: Lower logging level

2011-06-29  Daniel P. Berrange  

	Fix propagation of RPC errors from streams
	If a streams error is raised, virNetClientIOEventLoop
	returns 0, but an error is set. Check for this and
	propagate it if present

	* src/rpc/virnetclient.c: Propagate streams error

2011-06-29  Daniel P. Berrange  

	Fix crash when aborting a stream from a I/O callback
	If a callback being invoked from a stream issues a virStreamAbort
	operation, the stream data will be free'd but the callback will
	then still try to use this. Delay free'ing of the stream data when
	a callback is dispatching

	* src/fdstream.c: Delay stream free when callback is active

2011-06-29  Michal Privoznik  

	screenshot: Set access rights to temporary file
	Although we create a temporary file, it is owned by root:root and have
	rights 0600. In case qemu does not run under root, it is unable to write
	to that file and thus we transfer 0B sized file.

2011-06-29  Matthias Bolte  

	dnsmasq: Fix errno handling and don't unlink non-existing files
	addnhostsSave and hostsfileSave expect < 0 return value on error from
	addnhostsWrite and hostsfileWrite but then pass err instead of -err
	to virReportSystemError that expects an errno value.

	Also addnhostsWrite returns -ENOMEM and errno, change this to -errno.

	addnhostsWrite and hostsfileWrite tried to unlink the tempfile after
	renaming it, making both fail on the final step. Remove the unnecessary
	unlink calls.

2011-06-29  Eric Blake  

	maint: improve makefile whitespace
	None of these instances cause any semantic differences, but
	consistency is nice.

	* src/Makefile.am: Replace leading spaces with tabs.

2011-06-29  Osier Yang  

	tests: Fix memory leak in virnetmessagetest
	Detected when playing with "make -C tests valgrind".

	conf: Fix memory leak in virNetworkDNSDefFormat

2011-06-29  Matthias Bolte  

	network: Don't ignore errors in dnsmasq config file creation

2011-06-29  Matthias Bolte  

	network: Fix dnsmasq hostsfile creation logic and related tests
	networkSaveDnsmasqHostsfile was added in 8fa9c2214247 (Apr 2010).
	It has a force flag. If the dnsmasq hostsfile already exists force
	needs to be true to overwrite it. networkBuildDnsmasqArgv sets force
	to false, networkDefine sets it to true. This results in the
	hostsfile being written only in networkDefine in the common case.
	If no error occurred networkSaveDnsmasqHostsfile returns true and
	networkBuildDnsmasqArgv adds the --dhcp-hostsfile to the dnsmasq
	command line.

	networkSaveDnsmasqHostsfile was changed in 89ae9849f744 (24 Jun 2011)
	to return a new dnsmasqContext instead of reusing one. This change broke
	the logic of the force flag as now networkSaveDnsmasqHostsfile returns
	NULL on error, but the early return -- if force was not set and the
	hostsfile exists -- returns 0. This turned the early return in an error
	case and networkBuildDnsmasqArgv didn't add the --dhcp-hostsfile option
	anymore if the hostsfile already exists. It did because networkDefine
	created the hostsfile already.

	Then 9d4e2845d498 fixed the return 0 case in networkSaveDnsmasqHostsfile
	but didn't apply the force option correctly to the new addnhosts file.
	Now force doesn't control an early return anymore, but influences the
	handling of the hostsfile context creation and dnsmasqSave is always
	called now. This commit also added test cases that reveal several
	problems. First, the tests now calls functions that try to write the
	dnsmasq config files to disk. If someone runs this tests as root this
	might overwrite actively used dnsmasq config files, this is a no-go. Also
	the tests depend on configure --localstatedir, this needs to be fixed as
	well, because it makes the tests fail when localstatedir is different
	from /var.

	This patch does several things to fix this:

	1) Move dnsmasqContext creation and saving out of networkBuildDnsmasqArgv
	to the caller to separate the command line generation from the config
	file writing. This makes the command line generation testable without the
	risk of interfering with system files, because the tests just don't call
	dnsmasqSave.

	2) This refactoring of networkSaveDnsmasqHostsfile makes the force flag
	useless as the saving happens somewhere else now. This fixes the wrong
	usage of the force flag in combination with then newly added addnhosts
	file by removing the force flag.

	3) Adapt the wrong test cases to the correct behavior, by adding the
	missing --dhcp-hostsfile option. Both affected tests contain DHCP host
	elements but missed the necessary --dhcp-hostsfile option.

	4) Rename networkSaveDnsmasqHostsfile to networkBuildDnsmasqHostsfile,
	because it doesn't save the dnsmasqContext anymore.

	5) Move all directory creations in dnsmasq context handling code from
	the *New functions to dnsmasqSave to avoid directory creations in system
	paths in the test cases.

	6) Now that networkBuildDnsmasqArgv doesn't create the dnsmasqContext
	anymore the test case can create one with the localstatedir that is
	expected by the tests instead of the configure --localstatedir given one.

2011-06-29  Matthias Bolte  

	Fix compilation with systemtap 1.3
	Version 1.3 of  uses this macro

	  #define STAP_CAST(t) (size_t)t

	that breaks like this if t is a function

	  remote.c:1775: error: cast from function call of type 'const char *'
	  to non-matching type 'long unsigned int' [-Wbad-function-cast]

	For that to work it should probably look like this

	  #define STAP_CAST(t) ((size_t)(t))

	In systemtap 1.4 this was completely rewritten.

	Anyway, before commit df0b57a95a767c t was always a variable, but now
	also a function is used here, namely virNetSASLSessionGetIdentity.

	Use an intermediate variable to avoid this problem.

2011-06-28  Eric Blake  

	build: fix mingw build
	./autobuild.sh died on several messages resembling:

	../../src/rpc/virnetsocket.c: In function 'virNetSocketNewListenTCP':
	../../src/rpc/virnetsocket.c:231:9: error: implicit declaration of function 'bind_used_without_requesting_gnulib_module_bind' [-Wimplicit-function-declaration]
	../../src/rpc/virnetsocket.c:231:9: error: nested extern declaration of 'bind_used_without_requesting_gnulib_module_bind' [-Wnested-externs]

	Basically, gnulib socket fds are not safe to pass to mingw socket
	functions unless we pull in those gnulib modules.

	* bootstrap.conf (gnulib_modules): Add modules to handle socket
	functions on mingw.

2011-06-28  Eric Blake  

	sysinfo: fix parsing regression
	Detected by gcc -O2, introduced in commit 532ce9c2.  If dmidecode
	outputs a field unrecognized by the parsers, then the code would
	dereference an uninitialized eol variable.

	* src/util/sysinfo.c (virSysinfoParseBIOS)
	(virSysinfoParseSystem, virSysinfoParseProcessor)
	(virSysinfoParseMemory): Avoid uninitialized variable.

2011-06-28  Eric Blake  

	build: update translated files
	The last patch was incomplete.  The translated strings merely
	moved between generated file names, rather than disappearing.

	* cfg.mk (generated_files): Update generated file names.
	* po/POTFILES.in: Add remote_dispatch.h

2011-06-28  Jiri Denemark  

	build: Don't expect translatable strings in a dead file
	daemon/remote_dispatch_bodies.h is no longer with us and shouldn't be
	searched for translatable strings.

	daemon: Fix build without polkit

2011-06-28  Eric Blake  

	remote: fix uninitialized variable
	Detected by gcc -O2:

	remote/remote_driver.c: In function 'doRemoteOpen':
	remote/remote_driver.c:2753:26: error: 'sasl' may be used uninitialized in this function [-Werror=uninitialized]

	* src/remote/remote_driver.c (remoteAuthSASL): Initialize sasl.

2011-06-28  Eric Blake  

	build: fix 'make check' when pdwtags is available
	Problem introduced in commit 6818cf86.

	* src/remote_protocol-structs: Delete unused struct.

2011-06-28  Eric Blake  

	build: sanlock-devel is not in F15 yet
	* libvirt.spec.in (with_sanlock): Only default on in rawhide.
	Reported by Gerhard Stenzel.

2011-06-28  Daniel P. Berrange  

	Add documentation for configuration lock managers
	Add a page which documents how to configure lock managers,
	focusing on use of sanlock with the QEMU/KVM driver

	* docs/locking.html.in: Docs about lock managers
	* docs/sitemap.html.in: Add lock manager config to
	  the deployment section

2011-06-28  Daniel P. Berrange  

	Support automatic creation of leases for disks in sanlock
	The current sanlock plugin requires a central management
	application to manually add  elements to each guest,
	to protect resources that are assigned to it (eg writable
	disks). This makes the sanlock plugin useless for usage
	in more ad hoc deployment environments where there is no
	central authority to associate disks with leases.

	This patch adds a mode where the sanlock plugin will
	automatically create leases for each assigned read-write
	disk, using a md5 checksum of the fully qualified disk
	path. This can work pretty well if guests are using
	stable disk paths for block devices eg /dev/disk/by-path/XXXX
	symlinks, or if all hosts have NFS volumes mounted in
	a consistent pattern.

	The plugin will create one lockspace for managing disks
	with filename /var/lib/libvirt/sanlock/__LIBVIRT__DISKS__.
	For each VM disks, there will be another file to hold
	a lease /var/lib/libvirt/sanlock/5903e5d25e087e60a20fe4566fab41fd
	Each VM disk lease is usually 1 MB in size. The script
	virt-sanlock-cleanup should be run periodically to remove
	unused lease files from the lockspace directory.

	To make use of this capability the admin will need to do
	several tasks:

	 - Mount an NFS volume (or other shared filesystem)
	   on /var/lib/libvirt/sanlock
	 - Configure 'host_id' in /etc/libvirt/qemu-sanlock.conf
	   with a unique value for each host with the same NFS
	   mount
	 - Toggle the 'auto_disk_leases' parameter in qemu-sanlock.conf

	Technically the first step can be skipped, in which case
	sanlock will only protect against 2 vms on the same host
	using the same disk (or the same VM being started twice
	due to error by libvirt).

	* src/locking/libvirt_sanlock.aug,
	  src/locking/sanlock.conf,
	  src/locking/test_libvirt_sanlock.aug: Add config params
	  for configuring auto lease setup
	* libvirt.spec.in: Add virt-sanlock-cleanup program, man
	  page
	* tools/virt-sanlock-cleanup.in: Script to purge unused
	  disk resource lease files

2011-06-28  Daniel P. Berrange  

	Support loading a configuration file for sanlock plugin
	Introduce a configuration file with a single parameter
	'require_lease_for_disks', which is used to decide whether
	it is allowed to start a guest which has read/write disks,
	but without any leases.

	* libvirt.spec.in: Add sanlock config file and augeas
	  lens
	* src/Makefile.am: Install sanlock config file and
	  augeas lens
	* src/locking/libvirt_sanlock.aug: Augeas master lens
	* src/locking/test_libvirt_sanlock.aug: Augeas test file
	* src/locking/sanlock.conf: Example sanlock config
	* src/locking/lock_driver_sanlock.c: Wire up loading
	  of configuration file

2011-06-28  Daniel P. Berrange  

	Allow per-driver config file for lock manager plugins
	Allow a 'configFile' parameter to be passed into the lock
	drivers to provide configuration. Wire up the QEMU driver
	to pass in file names '/etc/libvirt/qemu-$NAME.conf
	eg qemu-sanlock.conf

	* src/locking/lock_driver.h, src/locking/lock_driver_nop.c,
	  src/locking/lock_driver_sanlock.c, src/locking/lock_manager.c,
	  src/locking/lock_manager.h: Add configFile parameter
	* src/qemu/qemu_conf.c: Pass in configuration file path to
	  lock driver plugins

2011-06-28  Laine Stump  

	network: add domain to unqualified names defined with 
	If a domain name is defined for a network, add the --expand-hosts
	option to the dnsmasq commandline. This results in the domain being
	added to any hostname that is defined in a dns  element and
	contains no '.' characters (i.e. it is an "unqualified"
	hostname). Since PTR records are automatically created for any name
	defined in , the result of a PTR request will change from the
	unqualified name to the qualified name.

	This also has the same effect on any hostnames that dnsmasq reads
	from the host's /etc/hosts file.

	(In the case of guest hostnames that were learned by dnsmasq via DHCP
	requests, they were already getting the domain name added on, even
	without --expand-hosts).

2011-06-28  Daniel P. Berrange  

	Cleanup remote protocol definitions
	The standard remote protocol for libvirtd no longer needs to
	include definitions of the generic message header/error structs
	or status codes. This is all defined in the generic RPC protocol

	* src/remote/remote_protocol.x: Remove all RPC message definitions
	* src/remote/remote_protocol.h, src/remote/remote_protocol.c:
	  Re-generate
	* daemon/remote_generate_stubs.pl: Delete obsolete script

2011-06-28  Daniel P. Berrange  

	Remove obsolete libvirtd mdns code
	libvirtd now uses the generic RPC code for MDNS, so its
	custom mdns APIs are no longer required

	* daemon/mdns.c, daemon/mdns.h: Removed obsolete files

2011-06-28  Daniel P. Berrange  

	Convert libvirtd over to the new RPC handling APIs
	This guts the libvirtd daemon, removing all its networking and
	RPC handling code. Instead it calls out to the new virServerPtr
	APIs for all its RPC & networking work

	As a fallout all libvirtd daemon error reporting now takes place
	via the normal internal error reporting APIs. There is no need
	to call separate error reporting APIs in RPC code, nor should
	code use VIR_WARN/VIR_ERROR for reporting fatal problems anymore.

	* daemon/qemu_dispatch_*.h, daemon/remote_dispatch_*.h: Remove
	  old generated dispatcher code
	* daemon/qemu_dispatch.h, daemon/remote_dispatch.h: New dispatch
	  code
	* daemon/dispatch.c, daemon/dispatch.h: Remove obsoleted code
	* daemon/remote.c, daemon/remote.h: Rewrite for new dispatch
	  APIs
	* daemon/libvirtd.c, daemon/libvirtd.h: Remove all networking
	  code
	* daemon/stream.c, daemon/stream.h: Update for new APIs
	* daemon/Makefile.am: Link to libvirt-net-rpc-server.la

2011-06-28  Daniel P. Berrange  

	Convert the remote driver to new RPC client APIs
	This guts the current remote driver, removing all its networking
	handling code. Instead it calls out to the new virClientPtr and
	virClientProgramPtr APIs for all RPC & networking work.

	* src/Makefile.am: Link remote driver with generic RPC code
	* src/remote/remote_driver.c: Gut code, replacing with RPC
	  API calls
	* src/rpc/gendispatch.pl: Update for changes in the way
	  streams are handled

2011-06-28  Daniel P. Berrange  

	Add XDR_CFLAGS to libvirt-net-rpc.la library
	* src/Makefile.am: Add XDR_CFLAGS

2011-06-28  Daniel P. Berrange  

	Ensure sanlock socket is labelled with the VM process label
	The libvirt sanlock plugin is intentionally leaking a file
	descriptor to QEMU. To enable QEMU to use this FD under
	SELinux, it must be labelled correctly. We dont want to use
	the svirt_image_t for this, since QEMU must not be allowed
	to actually use the FD. So instead we label it with svirt_t
	using virSecurityManagerSetProcessFDLabel

	* src/locking/domain_lock.c, src/locking/domain_lock.h,
	  src/locking/lock_driver.h, src/locking/lock_driver_nop.c,
	  src/locking/lock_driver_sanlock.c, src/locking/lock_manager.c,
	  src/locking/lock_manager.h: Optionally pass an FD back to
	  the hypervisor for security driver labelling
	* src/qemu/qemu_process.c: label the lock manager plugin
	  FD with the process label

2011-06-28  Daniel P. Berrange  

	Add a virSecurityManagerSetProcessFDLabel
	Add a new security driver method for labelling an FD with
	the process label, rather than the image label

	* src/libvirt_private.syms, src/security/security_apparmor.c,
	  src/security/security_dac.c, src/security/security_driver.h,
	  src/security/security_manager.c, src/security/security_manager.h,
	  src/security/security_selinux.c, src/security/security_stack.c:
	  Add virSecurityManagerSetProcessFDLabel & impl

2011-06-28  Daniel P. Berrange  

	Rename virSecurityManagerSetFDLabel method
	The virSecurityManagerSetFDLabel method is used to label
	file descriptors associated with disk images. There will
	shortly be a need to label other file descriptors in a
	different way. So the current name is ambiguous. Rename
	the method to virSecurityManagerSetImageFDLabel to clarify
	its purpose

	* src/libvirt_private.syms,
	  src/qemu/qemu_migration.c, src/qemu/qemu_process.c,
	  src/security/security_apparmor.c, src/security/security_dac.c,
	  src/security/security_driver.h, src/security/security_manager.c,
	  src/security/security_manager.h, src/security/security_selinux.c,
	  src/security/security_stack.c: s/FDLabel/ImageFDLabel/

2011-06-28  Matthias Bolte  

	Add node prefix to virNodeGet(CPU|Memory)Stats structs and defines

2011-06-28  Jiri Denemark  

	qemu: Fix update device for CURRENT + FORCE flags
	When CURRENT and FORCE flags were used together, UpdateDeviceFlags did
	nothing because it failed to transform CURRENT into either LIVE or
	CONFIG.

2011-06-27  Stefan Berger  

	nwfilter: Return error message about unresolvable variables
	This is in response to bugzilla 664629

	https://bugzilla.redhat.com/show_bug.cgi?id=664629

	The patch below returns an appropriate error message if the chain of
	nwfilters is found to contain unresolvable variables and therefore
	cannot be instantiated.

	Example: The following XMl added to a domain:

	    
	      
	      
	      
	      
	    

	that references the following filter

	
	  
	  
	

	now displays upon 'virsh start mydomain'

	error: Failed to start domain mydomain
	error: internal error Cannot instantiate filter due to unresolvable variable: DHCPSERVER

	'DHPCSERVER' is contained in allow-dhcp-server.

2011-06-27  Eric Blake  

	build: rename Vcpupin to VcpuPin
	We already have a public virDomainPinVcpu, which implies that
	Pin and Vcpu are treated as separate words.  Unreleased commit
	e261987c introduced virDomainGetVcpupinInfo as the first public
	API that used Vcpupin, although we had prior internal uses of
	that spelling.  For consistency, change the spelling to be two
	words everywhere, regardless of whether pin comes first or last.

	* daemon/remote.c: Treat vcpu and pin as separate words.
	* include/libvirt/libvirt.h.in: Likewise.
	* src/conf/domain_conf.c: Likewise.
	* src/conf/domain_conf.h: Likewise.
	* src/driver.h: Likewise.
	* src/libvirt.c: Likewise.
	* src/libvirt_private.syms: Likewise.
	* src/libvirt_public.syms: Likewise.
	* src/libxl/libxl_driver.c: Likewise.
	* src/qemu/qemu_driver.c: Likewise.
	* src/remote/remote_driver.c: Likewise.
	* src/xen/xend_internal.c: Likewise.
	* tools/virsh.c: Likewise.
	* src/remote/remote_protocol.x: Likewise.
	* src/remote_protocol-structs: Likewise.
	Suggested by Matthias Bolte.

2011-06-27  Minoru Usui  

	sysinfo: cleanup function/struct names.
	Fix lack of 'virSysinfo' prefix of functions/structs in src/util/sysinfo.[ch]

	sysinfo: fix lack of error check in virSysinfoFormat().
	Fix lack of error check in virSysinfoFormat().

2011-06-27  Matthias Bolte  

	tests: Partly fix networkxml2argvtest being configure result dependent
	Convert networkDnsmasqLeaseFileName to a replaceable function pointer
	that allow the testsuite to use a version of that function that is not
	depending on configure --localstatedir.

	This fixes 5 of 6 test failures, when configure --localstatedir isn't
	set to /var.

2011-06-27  Osier Yang  

	qemu: Remove bogus error codes for NUMA memory tuning
	This is no code between virSaveLastError and virGetLastError will
	set an error, remove the bogus codes.

2011-06-27  Laine Stump  

	network: fix indentation in networkBuildDnsmasqArgv
	This block was inadvertently added with the wrong indentation.

2011-06-27  Ruben Kerkhof  

	Only include parthelper if built with storage_disk
	Parthelper is only compiled if both --with-libvirtd
	and --with-storage-disk are set.

2011-06-25  Eric Blake  

	build: avoid uninitialized variable
	Caught by gcc -O2, during autobuild.sh.

	* src/qemu/qemu_driver.c (qemudDomainGetVcpupinInfo): Initialize vm.

2011-06-25  Eric Blake  

	build: fix VPATH builds
	The build currently fails when trying to create virnetprotocol.c
	into $(builddir)/rpc, which doesn't exist.  But since the file
	is part of the tarball, it should be generated into $(srcdir).
	Caught by autobuild.sh.

	* src/Makefile.am (VIR_NET_RPC_GENERATED): Generate into srcdir.

2011-06-25  Taku Izumi  

	vcpupin: add query option to virsh vcpupin command
	This patch teaches "virsh vcpupin" command to query if no list
	is given. Its feature is to show CPU affinity information in more
	reader-friendly way.

	 # virsh vcpupin VM --config
	 VCPU: CPU Affinity
	 ----------------------------------
	    0: 1-6,9-20
	    1: 10
	    2: 5,9-11,15-20
	    3: 1,3,5,7,9,11,13,15

	When cpulist is omitted, vcpu number is optional. When vcpu number is
	provided, information of only specified vcpu is displayed.

2011-06-25  Taku Izumi  

	vcpuinfo: add the code to fallback to try new API
	The "virsh vcpuinfo" command results in failure when the target domain
	is inactive on KVM. This patch improves this behavior by adding the
	fallback to invoke virDomainGetVcpupinInfo API in case of
	virDomainGetVcpus API returns error and the target domain is inactive.

	vcpupin: implement the remote protocol to address the new API
	This patch implements the remote protocol to address the new API
	(virDomainGetVcpupinInfo).

	vcpupin: implement the code to support new API for the qemu driver
	This patch implements the code to address the new API (virDomainGetVcpupinInfo)
	in the qemu driver.

2011-06-25  Taku Izumi  

	vcpupin: introduce the new libvirt API (virDomainGetVcpupinInfo)
	This patch introduces a new libvirt API (virDomainGetVcpupinInfo),
	as a counterpart to virDomainPinVcpuFlags.

	We can use virDomainGetVcpus API to retrieve CPU affinity information,
	but can't use this API against inactive domains (at least in case of KVM),
	as it lacks a flags parameter.
	The usual thing is to add a new virDomainGetVcpusFlags, but that API name
	is already occupied by the counterpart to virDomainGetMaxVcpus, which
	has a completely different signature.

	The virDomainGetVcpupinInfo is the new API to retrieve CPU affinity
	information of active and inactive domains.  While the usual convention
	is to list an array before its length, this API violates that rule
	in order to be more like virDomainGetVcpus (where maxinfo was doing
	double-duty as the length of two different arrays).

2011-06-25  Laine Stump  

	docs: fix indentation of sub-elements of  in network XML
	The sub-elements of  had been placed at the same level of
	indentation as ip itself, implying that they were really elements of
	. Within that, sub-elements of ip/dhcp were also at that same
	level. These have been double-indented.

	At the same time, I realized that the documentation for the new 
	element had been placed right in the middle of the description of the
	sub-elements of . I moved it up out of the way.

2011-06-25  Eric Blake  

	buf: protect against integer overflow
	It's unlikely that we'll ever want to escape a string as long as
	INT_MAX/6, but adding this check can't hurt.

	* src/util/buf.c (virBufferEscapeSexpr, virBufferEscapeString):
	Check for (unlikely) overflow.

2011-06-24  Eric Blake  

	remote: protect against integer overflow
	Integer overflow and remote code are never a nice mix.

	This has existed since commit 56cd414.

	* src/libvirt.c (virDomainGetVcpus): Reject overflow up front.
	* src/remote/remote_driver.c (remoteDomainGetVcpus): Avoid overflow
	on sending rpc.
	* daemon/remote.c (remoteDispatchDomainGetVcpus): Avoid overflow on
	receiving rpc.

2011-06-24  Eric Blake  

	build: update gnulib for intprops
	Done as a separate commit to make backporting the next patch easier.
	We are already using "intprops.h", but this makes it explicit.

	* .gnulib: Update, for syntax-check fix.
	* bootstrap.conf (gnulib_modules): Make intprops use explicit.
	* src/locking/domain_lock.c (includes): Drop unused header.
	* src/nwfilter/nwfilter_learnipaddr.c (includes): Use "", not <>,
	for gnulib.

2011-06-24  Eric Blake  

	build: avoid long line tests
	'make syntax-check' regression introduced in commit 60b9c69.

	* tests/networkxml2argvdata/*.argv: Break long lines.

2011-06-24  Matthias Bolte  

	Rename iface(G|S)etMacaddr to iface(G|S)etMacAddress for consistency

	Fix typo in libvirt_private.syms
	Triggered a linker error on MinGW.

2011-06-24  Michal Novotny  

	Network: Add support for DNS hosts definition to the network XML
	This commit introduces names definition for the DNS hosts file using
	the following syntax:

	  
	    
	      alias1
	      alias2
	    
	  

	Some of the improvements and fixes were done by Laine Stump so
	I'm putting him into the SOB clause again ;-)

2011-06-24  Michal Novotny  

	Network: Add additional hosts internal infrastructure

	Network: Add regression tests for the command-line arguments
	The regression testing done by comparison of command-line
	generated from the network XML file and the expected
	command-line arguments (read from file).

	Network: modify dnsmasq commandline build function to allow testing
	The dnsmasq commandline was being built as a part of running
	dnsmasq. This patch puts the commandline build into a separate
	function (and exports it as a private API) making it possible to build
	a dnsmasq commandline without executing it, so that we can write a
	test program to verify that the proper commandlines are being created.

2011-06-24  Michal Novotny  

	Add TXT record support for virtual DNS service
	This commit introduces the  element and  record for the
	virtual DNS network. The DNS TXT record can be defined using following
	syntax in the network XML file:

	  
	    
	  

	Also, the Relax-NG scheme has been altered to allow the texts without
	spaces only for the name element and some nitpicks about memory
	free'ing have been fixed by Laine so therefore I'm adding Laine to the
	SOB clause ;-)

2011-06-24  Matthias Bolte  

	python: Don't declare Py_ssize_t for Python 2.6
	Commit cd48c3f4e95597 added a Py_ssize_t typedef for Python < 2.7.
	But Py_ssize_t was added in Python 2.5. This makes the build fail
	for Python 2.6.

	Adjust the check to match Python < 2.5 to fix this.

2011-06-24  Matthias Bolte  

	tests: Improve IPv6 detection for virNetSocket tests
	getifaddrs can return an IPv6 address, but getaddrinfo can fail
	for an IPv6 address. Cover this combination.

2011-06-24  Jamie Strandboge  

	Move load of AppArmor profile to GenLabel()
	Commit 12317957ecd6c37a2fb16275dcdeeacfe25c517 introduced an incompatible
	architectural change for the AppArmor security driver. Specifically,
	virSecurityManagerSetAllLabel() is now called much later in
	src/qemu/qemu_process.c:qemuProcessStart(). Previously, SetAllLabel() was
	called immediately after GenLabel() such that after the dynamic label (profile
	name) was generated, SetAllLabel() would be called to create and load the
	AppArmor profile into the kernel before qemuProcessHook() was executed. With
	12317957ecd6c37a2fb16275dcdeeacfe25c517, qemuProcessHook() is now called
	before SetAllLabel(), such that aa_change_profile() ends up being called
	before the AppArmor profile is loaded into the kernel (via ProcessLabel() in
	qemuProcessHook()).

	This patch addresses the change by making GenLabel() load the AppArmor
	profile into the kernel after the label (profile name) is generated.
	SetAllLabel() is then adjusted to only reload_profile() and append stdin_fn to
	the profile when it is specified. This also makes the AppArmor driver work
	like its SELinux counterpart with regard to SetAllLabel() and stdin_fn.
	Bug-Ubuntu: https://launchpad.net/bugs/801569

2011-06-24  Eric Blake  

	docs: document correct flag name
	When adding virDomainGetVcpusFlags in commit ea3f5c6, I did
	enough rebasing that the doc comments in libvirt.c no longer
	matched the final chosen enum names in libvirt.h.

	And now we've gone ahead and deprecated the names
	VIR_DOMAIN_VCPU_{LIVE,CONFIG}.

	* src/libvirt.c (virDomainGetVcpusFlags): Fix comment.

2011-06-24  Eric Blake  

	Revert "Add new API virDomainBlockPull* to headers"
	This reverts commit 7d56a16d036d9aa7292e10e884c129742036f8a7.

	Conflicts:

		python/generator.py
		src/libvirt_public.syms

2011-06-24  Eric Blake  

	Revert "virDomainBlockPull: Implement the main entry points"
	This reverts commit 6419f596e114ee9f372136cc2b9eb19f1cbb7a77.

2011-06-24  Eric Blake  

	Revert "Add virDomainBlockPull support to the remote driver"
	This reverts commit d1693bb160ea78954592c45a40eb856190c619c8.

	Conflicts:

		daemon/remote.c
		src/remote/remote_driver.c
		src/remote/remote_protocol.x

2011-06-24  Eric Blake  

	Revert "Implement virDomainBlockPull for the qemu driver"
	This reverts commit 784ee08d2220d2a77424d5fc756b6eebe8efd5fa.

	Revert "Enable the virDomainBlockPull API in virsh"
	This reverts commit 3e2493ce28b7e20416e916fdf893a9569c267925.

2011-06-24  Eric Blake  

	Revert "Enable virDomainBlockPull in the python API."
	This reverts commit d74b86f5d6ecae3d18a391f90a918fcac75914db.

	Conflicts:

		python/generator.py

2011-06-24  Eric Blake  

	Revert "Asynchronous event for BlockPull completion"
	This reverts commit 12cd77a0c58a80179182f7d09e8e73f9f66b4677.

	Conflicts:

		python/libvirt-override-virConnect.py
		python/libvirt-override.c
		src/remote/remote_protocol.x

2011-06-24  Eric Blake  

	build: avoid python 2.4 build failure
	I'm not sure when Py_ssize_t was introduced; but Fedora 14 Python 2.7
	has it, while RHEL 5 Python 2.4 lacks it.  It should be easy enough
	to adjust if someone runs into problems.

	* python/typewrappers.h (Py_ssize_t): Define for older python.

2011-06-24  Eric Blake  

	build: fix NUMA build on RHEL 5
	Use NUMA's older nodemask_t (fixed-size map) rather than the newer
	'struct bitmask' (variable-size) in order to still compile on RHEL 5,
	with its numactl-devel-0.9.8.

	* src/qemu/qemu_process.c [HAVE_NUMA]: Prefer back-compat mode.
	(qemuProcessInitNumaMemoryPolicy): Use older nodemask_t.

2011-06-24  Minoru Usui  

	Add Memory Device Information to virSysinfoRead() from dmidecode type 17
	* src/util/sysinfo.[ch]: also parse and save all the SMBIOS informations
	  about memory modules

	Add Processor Information to virSysinfoRead() from dmidecode type 4
	* src/util/sysinfo.c: add parsing and formatting of processor
	  information data

	Cleanup virSysinfoRead()
	* src/util/sysinfo.c: Separate BIOSInfo and SystemInfo part from
	                      virSysinfoRead()

2011-06-24  Daniel P. Berrange  

	Make sure virnetprotocol.[ch] are in BUILT_SOURCES
	To ensure virnetprotocol.[ch] are generated before any other
	files, add them to BUILT_SOURCES and MAINTAINERCLEANFILES.
	At the same time, move ESX_DRIVER_GENERATED out of DISTCLEAN
	and into MAINTAINERCLEANFILES, since they are included in
	EXTRA_DIST

	* src/Makefile.am: Add virnetprotocol.[ch] to BUILT_SOURCES

2011-06-24  Eric Blake  

	maint: typo fixes
	* src/xenxs/xen_sxpr.c (xenParseSxprPCI): Comment fix.
	* tests/object-locking.ml (driverTables): Likewise.
	* tests/testutils.c (virtTestDifferenceBin): Likewise.

2011-06-24  Daniel P. Berrange  

	Fix some bugs in RPC protocol make rules
	The Makefile.am rules for generating RPC protocol had a couple
	of bugs

	 - A instance of remote/rpcgen_fix.pl  was not changed
	   to rpc/genprotocol.pl
	 - A dep from rpc/virnetmessage.h on the generated
	   rpc/virnetprotocol.h was missing
	 - The generated rpc/virnetprotocol.[ch] were not listed
	   in MAINTAINERCLEANFILES

	* Makefile.am: Fix RPC protocol generation

2011-06-24  Daniel P. Berrange  

	Add missing translation of error message from QEMU reboot patch
	* src/qemu/qemu_process.c: Add missing _(...)

2011-06-24  Daniel P. Berrange  

	Fix job type set in qemuMigrationPrepareDirect/PrepareTunnel
	The qemuMigrationPrepareDirect/PrepareTunnel methods accidentally
	set the domain job to  QEMU_JOB_MIGRATION_OUT when it should have
	been  QEMU_JOB_MIGRATION_IN. This didn't have any ill-effect, but
	it is none-the-less wrong.

	* src/qemu/qemu_migration.c: Fix job type

2011-06-24  Daniel P. Berrange  

	Avoid high privileges taint warning for QEMU session driver
	The code emitting taint warnings was mistakenly thinking
	that guests run from the QEMU session driver were tainted
	for having high privileges. This is of course nonsense
	since the session driver is always unprivileged

	* src/qemu/qemu_domain.c: Don't warn for high privileges in
	  non-privileged QEMU

2011-06-24  Daniel P. Berrange  

	Allow automatic kill of guests when a connection is closed
	If an application is using libvirt + KVM as a piece of its
	internal infrastructure to perform a specific task, it can
	be desirable to guarentee the VM dies when the virConnectPtr
	disconnects from libvirtd. This ensures the app can't leak
	any VMs it was using. Adding VIR_DOMAIN_START_AUTOKILL as
	a flag when starting guests enables this to be done.

	* include/libvirt/libvirt.h.in: All VIR_DOMAIN_START_AUTOKILL
	* src/qemu/qemu_driver.c: Support automatic killing of guests
	  upon connection close
	* tools/virsh.c: Add --autokill flag to 'start' and 'create'
	  commands

2011-06-24  Daniel P. Berrange  

	Automatically kill target QEMU if migration aborts abnormally
	Migration is a multi-step process

	  1. Begin(src)
	  2. Prepare(dst)
	  3. Perform(src)
	  4. Finish(dst)
	  5. Confirm(src)

	At step 2, a QEMU process is lauched in the destination to
	accept the incoming migration. Occasionally the process
	that is controlling the migration workflow aborts, and fails
	to call step 4, Finish. This leaves a QEMU process running
	on the target (albeit with paused CPUs). Unfortunately because
	step 2 actives a job on the QEMU process, it is unkillable by
	normal means.

	By registering the VM for autokill against the src virConnectPtr
	in step 2, we can ensure that the guest is forcefully killed off
	if the connection is closed without step 4 being invoked

	* src/qemu/qemu_migration.c: Register autokill in PrepareDirect
	  and PrepareTunnel. Unregister autokill on successful run
	  of Finish
	* src/qemu/qemu_process.c: Unregister autokill when stopping a
	  process

2011-06-24  Daniel P. Berrange  

	Add infrastructure to automatically destroy guests when a connection closes
	Sometimes it is useful to be able to automatically destroy a guest when
	a connection is closed. For example, kill an incoming migration if
	the client managing the migration dies. This introduces a map between
	guest 'uuid' strings and virConnectPtr objects. When a connection is
	closed, any associated guests are killed off.

	* src/qemu/qemu_conf.h: Add autokill hash table to qemu driver
	* src/qemu/qemu_process.c, src/qemu/qemu_process.h: Add APIs
	  for performing autokill of guests associated with a connection
	* src/qemu/qemu_driver.c: Initialize autodestroy map

2011-06-24  Daniel P. Berrange  

	Support reboots with the QEMU driver
	For controlled shutdown we issue a 'system_powerdown' command
	to the QEMU monitor. This triggers an ACPI event which (most)
	guest OS wire up to a controlled shutdown. There is no equiv
	ACPI event to trigger a controlled reboot. This patch attempts
	to fake a reboot.

	 - In qemuDomainObjPrivatePtr we have a bool fakeReboot
	   flag.
	 - The virDomainReboot method sets this flag and then
	   triggers a normal 'system_powerdown'.
	 - The QEMU process is started with '-no-shutdown'
	   so that the guest CPUs pause when it powers off the
	   guest
	 - When we receive the 'POWEROFF' event from QEMU JSON
	   monitor if fakeReboot is not set we invoke the
	   qemuProcessKill command and shutdown continues
	   normally
	 - If fakeReboot was set, we spawn a background thread
	   which issues 'system_reset' to perform a warm reboot
	   of the guest hardware. Then it issues 'cont' to
	   start the CPUs again

	* src/qemu/qemu_command.c: Add -no-shutdown flag if
	  we have JSON support
	* src/qemu/qemu_domain.h: Add 'fakeReboot' flag to
	  qemuDomainObjPrivate struct
	* src/qemu/qemu_driver.c: Fake reboot using the
	  system_powerdown command if JSON support is available
	* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
	  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
	  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
	  binding for system_reset command
	* src/qemu/qemu_process.c: Reset the guest & start CPUs if
	  fakeReboot is set

2011-06-24  Daniel P. Berrange  

	Move the RPC generator scripts into src/rpc
	Move the daemon/remote_generator.pl to src/rpc/gendispatch.pl
	and move the src/remote/rpcgen_fix.pl to src/rpc/genprotocol.pl

	* daemon/Makefile.am: Update for new name/location of generator
	* src/Makefile.am: Update for new name/location of generator

2011-06-24  Daniel P. Berrange  

	Introduce generic RPC client objects
	To facilitate creation of new clients using XDR RPC services,
	pull alot of the remote driver code into a set of reusable
	objects.

	 - virNetClient: Encapsulates a socket connection to a
	   remote RPC server. Handles all the network I/O for
	   reading/writing RPC messages. Delegates RPC encoding
	   and decoding to the registered programs

	 - virNetClientProgram: Handles processing and dispatch
	   of RPC messages for a single RPC (program,version).
	   A program can register to receive async events
	   from a client

	 - virNetClientStream: Handles generic I/O stream
	   integration to RPC layer

	Each new client program now merely needs to define the list of
	RPC procedures & events it wants and their handlers. It does
	not need to deal with any of the network I/O functionality at
	all.

2011-06-24  Daniel P. Berrange  

	Introduce generic RPC module for advertising via MDNS
	Allow RPC servers to advertise themselves using MDNS,
	via Avahi

	* src/rpc/virnetserver.c, src/rpc/virnetserver.h: Allow
	  registration of MDNS services via avahi
	* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add
	  API to fetch the listen port number
	* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add API to
	  fetch the local port number
	* src/rpc/virnetservermdns.c, src/rpc/virnetservermdns.h: Represent
	  an MDNS advertisement

2011-06-24  Daniel P. Berrange  

	Introduce generic RPC server objects
	To facilitate creation of new daemons providing XDR RPC services,
	pull a lot of the libvirtd daemon code into a set of reusable
	objects.

	 * virNetServer: A server contains one or more services which
	   accept incoming clients. It maintains the list of active
	   clients. It has a list of RPC programs which can be used
	   by clients. When clients produce a complete RPC message,
	   the server passes this onto the corresponding program for
	   handling, and queues any response back with the client.

	 * virNetServerClient: Encapsulates a single client connection.
	   All I/O for the client is handled, reading & writing RPC
	   messages.

	 * virNetServerProgram: Handles processing and dispatch of
	   RPC method calls for a single RPC (program,version).
	   Multiple programs can be registered with the server.

	 * virNetServerService: Encapsulates socket(s) listening for
	   new connections. Each service listens on a single host/port,
	   but may have multiple sockets if on a dual IPv4/6 host.

	Each new daemon now merely has to define the list of RPC procedures
	& their handlers. It does not need to deal with any network related
	functionality at all.

2011-06-24  Daniel P. Berrange  

	Integrate TLS/SASL directly into the socket APIs
	This extends the basic virNetSocket APIs to allow them to have
	a handle to the TLS/SASL session objects, once established.
	This ensures that any data reads/writes are automagically
	passed through the TLS/SASL encryption layers if required.

	* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Wire up
	  SASL/TLS encryption

2011-06-24  Daniel P. Berrange  

	Generic module for handling SASL authentication & encryption
	This provides two modules for handling SASL

	 * virNetSASLContext provides the process-wide state, currently
	   just a whitelist of usernames on the server and a one time
	   library init call

	 * virNetTLSSession provides the per-connection state, ie the
	   SASL session itself. This also include APIs for providing
	   data encryption/decryption once the session is established

	* src/Makefile.am: Add to libvirt-net-rpc.la
	* src/rpc/virnetsaslcontext.c, src/rpc/virnetsaslcontext.h: Generic
	  SASL handling code

2011-06-24  Daniel P. Berrange  

	Generic module for handling TLS encryption and x509 certs
	This provides two modules for handling TLS

	 * virNetTLSContext provides the process-wide state, in particular
	   all the x509 credentials, DH params and x509 whitelists
	 * virNetTLSSession provides the per-connection state, ie the
	   TLS session itself.

	The virNetTLSContext provides APIs for validating a TLS session's
	x509 credentials. The virNetTLSSession includes APIs for performing
	the initial TLS handshake and sending/recving encrypted data

	* src/Makefile.am: Add to libvirt-net-rpc.la
	* src/rpc/virnettlscontext.c, src/rpc/virnettlscontext.h: Generic
	  TLS handling code

2011-06-24  Daniel P. Berrange  

	Introduce a generic object for using network sockets
	Introduces a simple wrapper around the raw POSIX sockets APIs
	and name resolution APIs. Allows for easy creation of client
	and server sockets with correct usage of name resolution APIs
	for protocol agnostic socket setup.

	It can listen for UNIX and TCP stream sockets.

	It can connect to UNIX, TCP streams directly, or indirectly
	to UNIX sockets via an SSH tunnel or external command

	* src/Makefile.am: Add to libvirt-net-rpc.la
	* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Generic
	  sockets APIs
	* tests/Makefile.am: Add socket test
	* tests/virnetsockettest.c: New test case
	* tests/testutils.c: Avoid overriding LIBVIRT_DEBUG settings
	* tests/ssh.c: Dumb helper program for SSH tunnelling tests

2011-06-24  Daniel P. Berrange  

	Provide a simple object for encoding/decoding RPC messages
	This provides a new struct that contains a buffer for the RPC
	message header+payload, as well as a decoded copy of the message
	header. There is an API for applying a XDR encoding & decoding
	of the message headers and payloads. There are also APIs for
	maintaining a simple FIFO queue of message instances.

	Expected usage scenarios are:

	To send a message

	   msg = virNetMessageNew()

	   ...fill in msg->header fields..
	   virNetMessageEncodeHeader(msg)
	   ...loook at msg->header fields to determine payload filter
	   virNetMessageEncodePayload(msg, xdrfilter, data)
	   ...send msg->bufferLength worth of data from buffer

	To receive a message

	   msg = virNetMessageNew()
	   ...read VIR_NET_MESSAGE_LEN_MAX of data into buffer
	   virNetMessageDecodeLength(msg)
	   ...read msg->bufferLength-msg->bufferOffset of data into buffer
	   virNetMessageDecodeHeader(msg)
	   ...look at msg->header fields to determine payload filter
	   virNetMessageDecodePayload(msg, xdrfilter, data)
	   ...run payload processor

	* src/Makefile.am: Add to libvirt-net-rpc.la
	* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Internal
	  message handling API.
	* testutils.c, testutils.h: Helper for printing binary differences
	* virnetmessagetest.c: Validate all XDR encoding/decoding

2011-06-24  Daniel P. Berrange  

	Defines the basics of a generic RPC protocol in XDR
	This patch defines the basics of a generic RPC protocol in XDR.
	This is wire ABI compatible with the original remote_protocol.x.
	It takes everything except for the RPC calls / events from that
	protocol

	 - The basic header virNetMessageHeader (aka remote_message_header)
	 - The error object virNetMessageError  (aka remote_error)
	 - Two dummy objects virNetMessageDomain & virNetMessageNetwork
	   sadly needed to keep virNetMessageError ABI compatible with
	   the old remote_error

	The RPC protocol supports method calls, async events and
	bidirectional data streams as before

	* src/Makefile.am: Add rules for generating RPC code from
	  protocol & define a new libvirt-net-rpc.la helper library
	* src/rpc/virnetprotocol.x: New generic RPC protocol

2011-06-24  Eric Blake  

	build: avoid python 2.4 build failure
	On RHEL 5, I got:

	/usr/bin/python ./generator.py /usr/bin/python
	  File "./generator.py", line 427
	    "virStreamFree", # Needed in custom virStream __del__, but free shouldn't
	                   ^
	SyntaxError: invalid syntax

	* python/generator.py (function_skip_python_impl): Use same syntax
	as other skip lists.

2011-06-24  Eric Blake  

	build: avoid compiler failure
	GCC complained about a C99 for-loop declaration outside of C99 mode
	when compiling on RHEL 5.

	* src/qemu/qemu_driver.c (qemudDomainPinVcpuFlags): Avoid C99 for
	loop, since gcc 4.1.2 hates it.

2011-06-23  David S. Wang  

	documenting the 802.1Qbh parameters of a 'direct' interface
	This patch adds documentation about the 802.1Qbh related parameters
	of the virtualport element for 'direct' interfaces.

2011-06-23  Eric Blake  

	build: update to latest gnulib
	Gnulib has been busy, with 397 commits; it's easier to update now
	even without any known libvirt issue to be fixed, rather than
	having to analyze an even larger changeset later on.

	* .gnulib: Update to latest, for lots of changes.
	* bootstrap: Synchronize to upstream.

2011-06-23  Stefan Berger  

	build: fix compilation on systems missing libnl and new includes
	This patch fixes the compilation of netlink.c and interface.c on those
	systems missing either libnl or that have an older linux/if_link.h
	include file not supporting macvtap or VF_PORTS.

	WITH_MACVTAP is '1' if newer include files were detected, '0' otherwise.

	IFLA_PORT_MAX is defined in linux/if_link.h if yet more functionality is
	supported.

2011-06-23  Matthias Bolte  

	configure: Make libnl optional unless macvtap is required
	If macvtap is required then libnl has to be available, otherwise
	libnl is optional.

2011-06-23  Cole Robinson  

	python: Generate virStreamFree but don't expose in bindings
	Turns out I was right in removing this the first time :) This is
	needed in our custom __del__ function, but the C code wasn't
	being generated. Add new infrastructure to do what we want

2011-06-23  Eric Blake  

	maint: avoid future setgid problems
	* cfg.mk (sc_prohibit_setuid)
	(exclude_file_name_regexp--sc_prohibit_setuid): New rule.
	(VC_LIST_ALWAYS_EXCLUDE_REGEX): Always exempt po files.
	(exclude_file_name_regexp--sc_prohibit_asprintf): Simplify.
	(exclude_file_name_regexp--sc_prohibit_can_not): Drop.
	(exclude_file_name_regexp--sc_prohibit_doubled_word): Likewise.

2011-06-23  Matthias Bolte  

	docs: Make virConnectClose comply to apibuild.py expectations
	apibuild.py expects a sentence that starts with "Returns"
	describing the return value of a function.

2011-06-23  Osier Yang  

	util: Fix memory leak in virJSONParserHandleStartMap

	qemu: Fix memory leak in qemuProcessWaitForMonitor
	Move "VIR_FREE(buf) into label "closelog", so that "buf" could be
	freed before returning.

2011-06-23  Jiri Denemark  

	qemu: Only check for NUMA availability if required
	We only care about NUMA availability if NUMA configuration is requested
	in domain XML.

2011-06-23  Matthew Booth  

	storage: fix volDelete return when volume still being allocated
	volDelete used to return VIR_ERR_INTERNAL_ERROR when attempting to
	delete a volume which was still being allocated. It should return
	VIR_ERR_OPERATION_INVALID.

	* src/storage/storage_driver.c: Fix return of volDelete.

2011-06-23  Eric Blake  

	util: avoid PATH_MAX-sized array
	See previous patch for why this is good...

	* src/util/pci.c (struct _pciDevice, pciGetDevice, pciFreeDevice):
	Manage path dynamically.  Report snprintf overflow.
	* src/util/hostusb.c (struct _usbDevice, usbGetDevice)
	(usbFreeDevice): Likewise.

2011-06-23  Eric Blake  

	xen: avoid PATH_MAX-sized array
	See previous patch for why this is good...

	* src/xen/xen_driver.h (xenXMConfCache): Manage filename
	dynamically.
	* src/xen/xm_internal.c (xenXMConfigCacheAddFile)
	(xenXMConfigFree, xenXMDomainDefineXML): Likewise.

2011-06-23  Eric Blake  

	tests: avoid PATH_MAX-sized array
	See previous patch for why this is good...

	* src/test/test_driver.c (struct _testConn, testOpenFromFile)
	(testClose): Manage path dynamically.

2011-06-23  Eric Blake  

	virStorageVol: avoid PATH_MAX-sized array
	POSIX allows implementations where PATH_MAX is undefined, leading
	to compilation error.  Not to mention that even if it is defined,
	it is often wasteful in relation to the amount of data being stored.

	All clients of vol->key were audited, and found not to care about
	whether key is static or dynamic, except for these offenders:

	* src/datatypes.h (struct _virStorageVol): Manage key dynamically.
	* src/datatypes.c (virReleaseStorageVol): Free key.
	(virGetStorageVol): Copy key.

2011-06-22  Eric Blake  

	maint: add omitted copyright header
	Bug introduced in commit 6a597883.

	* src/util/netlink.h: Add boilerplate.

2011-06-22  Eric Blake  

	docs: fix docs to match behavior of virConnectClose
	* src/libvirt.c (virConnectClose): Mention reference count return.
	Reported by Michal Novotny, analyzed by Matthias Bolte.

2011-06-22  Stefan Berger  

	cleanup: make several interface functions commonly available
	In a second cleanup step this patch makes several interface functions from macvtap.c commonly available by moving them into interface.c and prefixing their names with 'iface'. Those functions taking Linux-specific structures as parameters are only visible on Linux.

	ifaceRestoreMacAddress returns the return code from the ifaceSetMacAddr call and display an error message if setting the MAC address did not work. The caller is unchanged and still ignores the return code (which is ok).

2011-06-22  Stefan Berger  

	cleanup: make nlComm commonly available
	In a first cleanup step, make nlComm from macvtap.c commonly available
	for other code to use. Since nlComm uses Linux-specific structures as
	parameters it's prototype is only visible on Linux.

2011-06-22  Eric Blake  

	maint: remove syntax-check exception
	We weren't using the @FOO@ notation for a Makefile substitution,
	but instead for a sed rule, so using [@]FOO@ instead avoids the
	need to exempt this syntax check.

	* cfg.mk (_makefile_at_at_check_exceptions): Delete.
	* tools/Makefile.am (virt-xml-validate, virt-pki-validate): Avoid
	tripping syntax-check.
	Reported by Daniel P. Berrange.

2011-06-22  Daniel P. Berrange  

	Fix use-after-free in handling domain taint flags
	* src/conf/domain_conf.c: Fix use after free

2011-06-22  Daniel P. Berrange  

	Remove macvtap dependency on domain configuration
	Files under src/util must not depend on src/conf
	Solve the macvtap problem by moving the definition
	of macvtap modes from domain_conf.h into macvtap.h

	* src/util/macvtap.c, src/util/macvtap.h: Add enum
	  for macvtap modes
	* src/conf/domain_conf.c, src/conf/domain_conf.h: Remove
	  enum for macvtap modes

2011-06-22  Osier Yang  

	util: Correct the error prompt string
	virCommandProcessIO: It's reading from stdout or stderr of child,
	but not writing.

2011-06-22  Osier Yang  

	qemu: Fix one memory leak
	* src/qemu/qemu_domain.c: (qemuDomainAppendLog)

	  Free "message" in "cleanup".

2011-06-22  Daniel Veillard  

	Fix to python API extractor and API doc generation
	This fixes a number of issues most of them raised by Eric Blake on the
	generated documentation output:
	   - parsing of "long long int" and similar
	   - add parsing of unions within a struct
	   - remove spurious " * " fron comments on structure fields and enums
	   - fix concatenation of base type and name in arrays
	   - extend XSLT to cope with union in structs

	* docs/apibuild.py: fix and extend API extraction tool
	* docs/newapi.xsl: extend the stylesheets to cope with union in
	  public structures

2011-06-22  Michal Privoznik  

	qemu: domain I/O asynchronous handling
	For virtio disks and interfaces, qemu allows users to enable or disable
	ioeventfd feature. This means, qemu can execute domain code, while
	another thread waits for I/O event. Basically, in some cases it is win,
	in some loss. This feature is available via 'ioeventfd' attribute in disk
	and interface  element. It accepts 'on' and 'off'. Leaving this
	attribute out defaults to hypervisor decision.

2011-06-21  Eric Blake  

	build: require newer netcf when it is available
	When building rpms for newer Fedora or RHEL, take advantage of the
	newer netcf packaging to guarantee interface snapshot support.

	* libvirt.spec.in (BuildRequires): Bump minimum version on
	platforms that support netcf 0.1.8.

2011-06-21  Eric Blake  

	virsh: avoid bogus description
	https://bugzilla.redhat.com/show_bug.cgi?id=682121

	Gettext reserves the empty string for internal use, and it must
	not be passed through _().  We were violating this for commands
	that (for whatever reason) used "" for their description.

	* tools/virsh.c (vshCmddefHelp): Don't translate empty string.
	Reported by Tatsuo Kawasaki.

2011-06-21  Dirk Herrendoerfer  

	set and restore MAC address of a NIC when using PASSTHROUGH mode
	The following patch addresses the problem that when a PASSTHROUGH
	mode DIRECT NIC connection is made the MAC address of the NIC is
	not automatically set and reset to the configured VM MAC and
	back again.

	The attached patch fixes this problem by setting and resetting the MAC
	while remembering the previous setting while the VM is running.
	This also works if libvirtd is restarted while the VM is running.

	the patch passes make syntax-check

2011-06-21  Cole Robinson  

	python: Mark event callback wrappers as private
	These functions aren't intended to be called directly by users, so mark
	them as private.

	While we're at it, remove unneeded exception handling, and break some
	long lines.

2011-06-21  Cole Robinson  

	python: events: Fix C->Python handle callback prototype
	If registering our own event loop implementation written in python,
	any handles or timeouts callbacks registered by libvirt C code must
	be wrapped in a python function. There is some argument trickery that
	makes this all work, by wrapping the user passed opaque value in
	a tuple, along with the callback function.

	Problem is, the current setup requires the user's event loop to know
	about this trickery, rather than just treating the opaque value
	as truly opaque.

	Fix this in a backwards compatible manner, and adjust the example
	python event loop to do things the proper way.

2011-06-21  Cole Robinson  

	python: Add bindings for virEvent*Handle/Timeout

	events: Correct virEventAddTimeout docs

2011-06-21  Cole Robinson  

	Promote virEvent*Handle/Timeout to public API
	Since we virEventRegisterDefaultImpl is now a public API, callers need
	a way to invoke the default registered Handle and Timeout functions. We
	already have general functions for these internally, so promote
	them to the public API.

	v2:
	    Actually add APIs to libvirt.h

2011-06-21  Cole Robinson  

	python: Implement virStreamSend/RecvAll helpers
	Pure python implementation. The handler callbacks have been altered
	a bit compared to the C API: RecvAll doesn't pass length of the data read
	since that can be trivially obtained from python string objects, and SendAll
	requires the handler to return the string data to send rather than
	store the data in a string pointer.

	python: Implement virStreamSend/Recv
	The return values for the python version are different that the C version
	of virStreamSend: on success we return a string, an error raises an exception,
	and if the stream would block we return int(-2). We need to do this
	since strings aren't passed by reference in python.

2011-06-21  Osier Yang  

	conf: Make full use of goto label
	* virDomainDefParse: There is a goto label "no_memory", which
	reports OOM error, and then fallthrough label "error". This
	patch changes things like following:

	    virReportOOMError();
	    goto error;

	into:

	    goto no_memory;

2011-06-21  Osier Yang  

	conf: Fix one memory leak
	Free def->numatune.memory.nodemask in virDomainDefFree.

2011-06-20  Cole Robinson  

	python: Implement bindings for virStreamEventAddCallback
	v2:
	    Don't generate virStreamFree

	python: generator: Don't print warning if nothing to warn about

2011-06-20  Cole Robinson  

	python: libvirt-override: use simpler debug
	In a couple instances we have to mark a debug variable as ATTRIBUTE_UNUSED
	to avoid warnings.

	v2:
	    Use #if 0 to comment out debug define

2011-06-20  Eric Blake  

	virsh: enhance snapshot-create-as
	Similar to pool-create-as.

	* tools/virsh.c (cmdSnapshotCreateAs): Add --print-xml.
	* tools/virsh.pod: Document it.

2011-06-20  Eric Blake  

	virsh: add snapshot-create-as command
	Producing an xml file just for name and description fields is
	overkill; this makes life easier from virsh.

	* tools/virsh.c (cmdSnapshotCreateAs): New command.
	(snapshotCmds): Install it.
	* tools/virsh.pod: Document it.

2011-06-20  Eric Blake  

	virsh: clarify snapshot vs. save
	* tools/virsh.c (info_snapshot_create, info_save): Clarify
	description.
	* tools/virsh.pod (save): Likewise.

2011-06-20  Daniel P. Berrange  

	Fix compile warnings in virsh vcpupin
	The 'char *cur' variable was being assigned from a
	'const char *' string, thus discarding constness.
	As well as causing a compile warning, it masked a
	piece of code which attempts to assign to the
	previously const string.

	* tools/virsh.c: Fix const-ness of 'cur' variable in vcpupin

2011-06-20  Matthias Bolte  

	remote generator: Allow to annotate arrays with typecasts
	Removes special case code from the generator and handle additional
	methods.

	The generated version of remoteDispatchDomainPinVcpu(Flags) has no
	length check, but this check was useless anyway as it was applied to
	data that was already deserialized from its XDR form.

2011-06-20  Matthias Bolte  

	remote: Handle functions that return optional strings

	remote: Generate virDomainGetBlockPullInfo
	It was already generatable but skipped.

2011-06-20  Taku Izumi  

	vcpupin: add vcpupin resetting feature to qemu driver
	Pinning to all physical cpus means resetting, hence it is preferable to
	delete vcpupin setting of XML.

	This patch changes qemu driver to delete vcpupin setting by invoking
	virDomainVcpupinDel API when pinning the specified virtual cpu to
	all host physical cpus.

2011-06-20  Taku Izumi  

	vcpupin: add virDomainVcpupinDel function
	This patch add the private API (virDomainVcpupinDel).
	This API can delete the vcpupin setting of a specified virtual cpu.

2011-06-20  Taku Izumi  

	vcpupin: add reset option to virsh vcpupin command
	When resetting vcpupin setting, we have to specify all host physical
	cpus as a cpulist parameter of virsh vcpupin command. It's a little
	tedious.

	This patch changes to allow to receive the special keyword 'r' as a cpulist
	parameter of virsh vcpupin command when resetting vcpupin setting.

	If you set the following:

	 # virsh vcpupin VM 0 r

	the vcpu0 will be pinned to all physical cpus.

2011-06-20  Taku Izumi  

	vcpupin: improve vcpupin definition of virsh vcpupin
	When using vcpupin command, we have to speficy comma-separated list as cpulist,
	but this is tedious in case the number of phsycal cpus is large.
	This patch improves this by introducing special markup "-" and "^" which are
	similar to XML schema of "cpuset" attribute.

	The example:

	 # virsh vcpupin Guest 0 0-15,^8

	 is identical to

	 # virsh vcpupin Guest 0 0,1,2,3,4,5,6,7,9,10,11,12,13,14,15

	NOTE: The expression is sequentially evaluated, so "0-15,^8" is not identical
	to "^8,0-15".

2011-06-20  Matthias Bolte  

	Fix preprocessor indentation in nodeinfo.c

	Move XenAPI driver to correct spec file section
	The XenAPI driver works like the ESX and PHyp driver by using its
	own HTTPS based remote protocol.

2011-06-20  Osier Yang  

	numatune: Add tests for numatune XML
	Only add tests for qemuxmlargvtest.c, as there is no qemu command line
	for numatune XML, just want to make sure the XML could be validated
	well.

2011-06-20  Osier Yang  

	numatune: Support NUMA memory tuning in qemu driver
	Implemented as setting NUMA policy between fork and exec as a hook,
	using libnuma. Only support memory tuning on domain process currently.

	For the nodemask out of range, will report soft warning instead of
	hard error in libvirt layer. (Kernel will be silent as long as one
	of set bit in the nodemask is valid on the host. E.g. For a host
	has two NUMA nodes, kernel will be silent for nodemask "01010101").
	So, soft warning is the only thing libvirt can do, as one might want
	to specify the numa policy prior to a node that doesn't exist yet,
	however, it may come as hotplug soon.

2011-06-20  Osier Yang  

	numatune: Support persistent XML for numatune
	* src/conf/domain_conf.h: Introduce one new struct for representing
	NUMA tuning related stuffs.

	* src/conf/domain_conf.c: Parse and format numatune XML.

2011-06-20  Osier Yang  

	numatune: Add doc for new numatune XML

2011-06-20  Osier Yang  

	numatune: Define XML schema
	Example XML:

	
	  
	

	Please enter the commit message for your changes. Lines starting

2011-06-20  Hu Tao  

	Update qemuDomainSetBlkioParameters to use flags

	update qemuDomainGetBlkioParameters to use flags

	Add new parameters for blkiotune
	Add --config, --live and --current for command blkiotune

2011-06-20  Wen Congyang  

	build: fix building error when building without libvirtd
	When building libvirt without libvirtd, we will receive the following error
	message:

	make[3]: Entering directory `/home/wency/rpmbuild/BUILD/libvirt-0.9.2/tools'
	  CC     virsh-virsh.o
	  CC     virsh-console.o
	  GEN    virt-xml-validate
	  GEN    virt-pki-validate
	  CCLD   virsh
	./src/.libs/libvirt.so: undefined reference to `numa_available'
	./src/.libs/libvirt.so: undefined reference to `numa_max_node'
	collect2: ld returned 1 exit status

	The reason is that: we check numactl only when building qemu driver, and qemu
	driver will not be built when bulding without libvirtd. So with_numactl's
	value is check and we will not link libnuma.so.

	In the other function, we call numa_available() and numa_max_node() only
	when HAVE_NUMACTL is 1. We should do the same check in the function nodeGetMemoryStats().

2011-06-20  Jamie Strandboge  

	apparmor: implement AppArmorSetFDLabel()
	During a savevm operation, libvirt will now use fd migration if qemu
	supports it. When the AppArmor driver is enabled, AppArmorSetFDLabel()
	is used but since this function simply returns '0', the dynamic AppArmor
	profile is not updated and AppArmor blocks access to the save file. This
	patch implements AppArmorSetFDLabel() to get the pathname of the file by
	resolving the fd symlink in /proc, and then gives that pathname to
	reload_profile(), which fixes 'virsh save' when AppArmor is enabled.

	Reference: https://launchpad.net/bugs/795800

2011-06-17  Daniel P. Berrange  

	Add missing checks for NULL in domain lock manager
	The domain lock manager forgot to include a bunch of checks
	for NULL which could occur on OOM

	* src/locking/domain_lock.c: Add checks for NULL

2011-06-17  Daniel P. Berrange  

	Fix errno return in safezero()
	Most of the safezero() implementations return -1 on error,
	setting errno. The safezero() impl using posix_fallocate()
	though returned a positive errno value on error (due to
	the unusual API contract of posix_fallocate() compared to
	most syscall APIs).

	* src/util/util.c: Ensure safezero() returns -1 and sets
	  errno on error.
	* src/storage/storage_backend.c: Change safezero != 0 to
	  < 0 for detecting errors

2011-06-17  Daniel P. Berrange  

	Add missing error reporting when loading mac filter config for QEMU
	If the 'mac_filter' configuration parameter is enabled, and there
	is a failure to enable filtering, no error is reported back to
	the caller. Also fix some bogus whitespace indentation for
	hugetlbfs_mount

	* src/qemu/qemu_conf.c: Add missing error reporting

2011-06-16  Jiri Denemark  

	virsh: Add support for virDomainGetControlInfo

	qemu: Implement virDomainGetControlInfo

	Wire protocol and remote driver for virDomainGetControlInfo

	Introduce virDomainGetControlInfo API
	The API can be used to query current state of an interface to VMM used
	to control a domain. In QEMU world this translates into monitor
	connection.

2011-06-16  Matthias Bolte  

	Fix documentation of virStreamRecv
	virStreamRecv is for reading.

	Also add some missing punctuation to virStreamSend's documentation.

2011-06-15  Eric Blake  

	sendkey: use consistent API convention
	Even though rpc uses 'unsigned int' for the _len parameter that
	passes the length of item, the public libvirt APIs all
	use 'int' and filter out lengths < 0, except for virDomainSendKey.

	* include/libvirt/libvirt.h.in (virDomainSendKey): All other APIs
	use int for array length.
	* src/libvirt.c (virDomainSendKey): Adjust.
	* src/driver.h (virDrvDomainSendKey): Likewise.
	* daemon/remote_generator.pl: Likewise.

2011-06-15  Eric Blake  

	build: export correct function names
	Detected by autobuild.sh, when targetting mingw.
	Introduced in commit 98bfdff.

	* src/libvirt_private.syms: Fix typos.

2011-06-15  Eric Blake  

	build: avoid compiler warning on non-Linux
	Detected by autobuild.sh when cross-building for mingw.
	Introduced in commits ce76e85 and af35cec.

	* src/nodeinfo.c (nodeGetCPUStats, nodeGetMemoryStats): Mark
	parameters as potentially unused.

2011-06-15  Matthias Bolte  

	remote generator: Handle struct returning functions better (part 2)
	Commit 64000eabedf2 is part 1, that only covered the daemon side by
	accident. Part 2 covers the client side too.

	Fix apibuild.py warnings about missing comment headers
	Also improve wording of some comments.

	remote generator: Handle struct returning functions better
	The position of the struct parameter in the function signature
	differs. Instead of hardcoding the handling for this add an annotation
	to the .x file to define the position.

2011-06-15  Daniel Gollub  

	support for Xen HVM Viridian (Hyper-V) enlightenment interface
	Introduce libvirt support for Xen HVM Viridian (Hyper-V) enlightenment
	interface guest feature.

	 src/conf/domain_conf.c     |    3 ++-
	 src/conf/domain_conf.h     |    1 +
	 src/xen/xen_hypervisor.c   |   11 +++++++++++
	 src/xenapi/xenapi_driver.c |    2 ++
	 src/xenapi/xenapi_utils.c  |    2 ++
	 src/xenxs/xen_sxpr.c       |    4 ++++
	 src/xenxs/xen_xm.c         |   12 +++++++++++-
	 7 files changed, 33 insertions(+), 2 deletions(-)

2011-06-15  Eric Blake  

	virsh: reduce complexity in argv iteration
	This reduces things from O(n^2) to O(n).

	* tools/virsh.c (vshCommandOptArgv): Change signature.
	(cmdEcho): Update caller.
	Based on a patch by Lai Jiangshan.

2011-06-15  Daniel P. Berrange  

	Add support for network filter code in LXC driver
	The LXC driver networking uses veth device pairs. These can
	be easily hooked into the network filtering code.

	* src/lxc/lxc_driver.c: Add calls to setup/teardown nwfilter

2011-06-15  Daniel P. Berrange  

	Fix allocation of veth's to not skip an index
	The algorithm for autoassigning vethXXX devices, was always
	skipping over the starting dev index when finding a free
	name for the guest device. This should only be done if the host
	device was autoallocated.

	* src/lxc/veth.c: Don't skip over veth indexes

2011-06-15  Jiri Denemark  

	qemu: Translate boot config into bootindex if possible
	Prefer bootindex=N option for -device over the old way -boot ORDER
	possibly accompanied with boot=on option for -drive. This gives us full
	control over which device will actually be used for booting guest OS.
	Moreover, if qemu doesn't support boot=on, this is the only way to boot
	of certain disks in some configurations (such as virtio disks when used
	together IDE disks) without transforming domain XML to use per device
	boot elements.

2011-06-15  Adam Litke  

	Asynchronous event for BlockPull completion
	When an operation started by virDomainBlockPullAll completes (either with
	success or with failure), raise an event to indicate the final status.  This
	allows an API user to avoid polling on virDomainBlockPullInfo if they would
	prefer to use the event mechanism.

	* daemon/remote.c: Dispatch events to client
	* include/libvirt/libvirt.h.in: Define event ID and callback signature
	* src/conf/domain_event.c, src/conf/domain_event.h,
	  src/libvirt_private.syms: Extend API to handle the new event
	* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
	  for block_stream completion and emit a libvirt block pull event
	* src/remote/remote_driver.c: Receive and dispatch events to application
	* src/remote/remote_protocol.x: Wire protocol definition for the event
	* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
	  src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
	  from QEMU monitor

2011-06-15  Adam Litke  

	Enable virDomainBlockPull in the python API.
	virDomainBlockPullAll and virDomainBlockPullAbort are handled automatically.
	virDomainBlockPull and virDomainBlockPullInfo require manual overrides since
	they return a custom type.

	* python/generator.py: reenable bindings for this entry point
	* python/libvirt-override-api.xml python/libvirt-override.c:
	  manual overrides

	Acked-by: Daniel P. Berrange 

2011-06-15  Adam Litke  

	Enable the virDomainBlockPull API in virsh
	Define two new virsh commands:
	 * blockpull: Perform block pull operations (incremental plus start
	              and stop continuous streams)
	 * blockpullinfo: Retrieve progress info for continuous block pull

	Share print_job_progress() with the migration code.

	* tools/virsh.c: implement the new commands

2011-06-15  Adam Litke  

	Implement virDomainBlockPull for the qemu driver
	The virDomainBlockPull* family of commands are enabled by the
	'block_stream' and 'info block_stream' qemu monitor commands.

	* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk
	  streaming by using the stream and info stream text monitor commands
	* src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor

	Acked-by: Daniel P. Berrange 

2011-06-15  Adam Litke  

	Add virDomainBlockPull support to the remote driver
	The generator can handle DomainBlockPullAll and DomainBlockPullAbort.
	DomainBlockPull and DomainBlockPullInfo must be written by hand.

	* src/remote/remote_protocol.x: provide defines for the new entry points
	* src/remote/remote_driver.c daemon/remote.c: implement the client and
	  server side
	* src/remote_protocol-structs: structure definitions for protocol verification

2011-06-15  Adam Litke  

	virDomainBlockPull: Implement the main entry points
	* src/libvirt.c: implement the main entry points

	Acked-by: Daniel P. Berrange 

2011-06-15  Adam Litke  

	Add new API virDomainBlockPull* to headers
	Set up the types for the block pull functions and insert them into the
	virDriver structure definition.  Symbols are exported in this patch to prevent
	documentation compile failures.

	* include/libvirt/libvirt.h.in: new API
	* src/driver.h: add the new entry to the driver structure
	* python/generator.py: fix compiler errors, the actual python bindings are
	  implemented later
	* src/libvirt_public.syms: export symbols

2011-06-15  Marc-André Lureau  

	spice: add  option
	From a security pov copy and paste between the guest and the client is not
	always desirable. So we need to be able to enable/disable this. The best place
	to do this from an administration pov is on the hypervisor, so the qemu cmdline
	is getting a spice disable-copy-paste option, see bug 693645. Example qemu
	invocation:
	qemu -spice port=5932,disable-ticketing,disable-copy-paste

	https://bugzilla.redhat.com/show_bug.cgi?id=693661

2011-06-15  Minoru Usui  

	virNodeGetMemoryStats: Implement linux support

	virNodeGetMemoryStats: Implement virsh support

	virNodeGetMemoryStats: Implement remote protocol

	virNodeGetMemoryStats: Implement public API

	virNodeGetMemoryStats: Define internal driver API

	virNodeGetMemoryStats: Expose new API

2011-06-14  Minoru Usui  

	virNodeGetCPUStats: Implement linux support

	virNodeGetCPUStats: Implement virsh support

	virNodeGetCPUStats: Implement remote protocol

	virNodeGetCPUTime: Implement public API

	virNodeGetCPUStats: Define internal driver API

	virNodeGetCPUStats: Expose new API

2011-06-14  Guido Günther  

	Skip nodeinfo test on non intel architectures
	since the testfiles assume a /proc/cpuinfo specific to this
	architecture. We e.g. can't parse the number of cores on other
	architectures.

2011-06-14  Michael Chapman  

	Fix autostart flag when loading running domains
	Drivers load running persistent and transient domain configs before
	inactive persistent domain configs, however only the latter would set a
	domain's autostart flag. This mismatch between the loaded and on-disk
	state could later cause problems with "virsh autostart":

	  # virsh autostart example
	  error: Failed to mark domain example as autostarted
	  error: Failed to create symlink '/etc/libvirt/qemu/autostart/example.xml to '/etc/libvirt/qemu/example.xml': File exists

	This patch ensures the autostart flag is set correctly even when the
	domain is already defined.

	Fixes:

	  https://bugzilla.redhat.com/show_bug.cgi?id=632100
	  https://bugzilla.redhat.com/show_bug.cgi?id=675319

2011-06-14  Lai Jiangshan  

	send-key: Implementing the remote protocol

	send-key: Implementing the public API

	send-key: Defining the internal API

2011-06-14  Lai Jiangshan  

	send-key: Defining the public API
	Add public virDomainSendKey() and enum libvirt_keycode_set
	for the @codeset.

	Python version of virDomainSendKey() has not been implemented yet,
	it will be done soon.

2011-06-14  Lai Jiangshan  

	remote generator: Handle (unsigned) int arrays

2011-06-14  Lai Jiangshan  

	add VSH_OFLAG_REQ_OPT options
	A VSH_OFLAG_REQ_OPT option means --optionname is required when used.
	It will kill any ambiguity, even a !VSH_OFLAG_REQ option listed before
	a VSH_OFLAG_REQ option, if the !VSH_OFLAG_REQ option is a
	VSH_OFLAG_REQ_OPT option.

	It will help us use optional argument with VSH_OT_ARGV argument.

2011-06-14  Lai Jiangshan  

	allow name for VSH_OT_ARGV options
	A name will improve the usege, example

	# virsh help echo
	  NAME
	    echo - echo arguments

	  SYNOPSIS
	    echo [--shell] [--xml] []...

	  DESCRIPTION
	    Echo back arguments, possibly with quoting.

	  OPTIONS
	    --shell          escape for shell use
	    --xml            escape for XML use
	             arguments to echo

	"[]..." is added to SYNOPSIS.
	"         arguments to echo" is added to OPTIONS.

2011-06-14  Ohad Levy  

	docs: updated list of applications using libvirt.
	Added:
	* Virt ruby bindings
	* Foreman for provisioning and webui.

2011-06-14  Eric Blake  

	storage: avoid an intermediate malloc
	Suggested here:
	https://www.redhat.com/archives/libvir-list/2011-May/msg00594.html

	* src/storage/storage_backend.c (virStorageBackendCreateQemuImg):
	Generate size inline.

2011-06-14  Eric Blake  

	command: avoid double close
	Previously, the parent process opened 'null' to /dev/null, then
	the child process closes 'null' as well as 'childout'.  But if
	childout was set to be null, then this is a double close.  At
	least the double close was confined to the child process after a
	fork, and therefore there is no risk of another thread opening
	an fd of the same value to be bitten by the double close, but it
	is always better to avoid double-close to begin with.

	Additionally, if all three fds were specified, then opening
	'null' was wasted.

	This patch fixes things to lazily open null on the first use,
	then guarantees it gets closed exactly once.

	* src/util/command.c (getDevNull): New helper function.
	(virExecWithHook): Use it to avoid spurious opens and double close.

2011-06-14  Eric Blake  

	command: reduce duplicated debug messages
	This also reduces malloc pressure for invoking a child when
	VIR_DEBUG is enabled.

	* src/util/command.c (virExecWithHook): Drop debug, since the only
	caller (virCommandRunAsync) also prints debug info.

2011-06-14  Eric Blake  

	docs: improve VIR_DOMAIN_AFFECT_CURRENT description
	* include/libvirt/libvirt.h.in (virDomainModificationImpact): Reword.

2011-06-14  Cole Robinson  

	qemu: Fix parsing 'info chardev'
	If qemu supports -chardev, our char frontend aliases are ex. 'charserial0'
	not just 'serial0'. Typically we don't use this code path because the
	pty's are scraped from stdout.

	qemu: Scrape stdout for virtio console pty
	Currently we forget to do this and have to fallback to info chardev (which
	also fails, see following patch)

2011-06-14  Michal Privoznik  

	virsh: Add daemon version reporting
	'virsh version' might report against which version of libvirtd is
	running.

2011-06-14  Guido Günther  

	nodeinfo: remove superflous braces
	that break compilation on non intel architectures:

	mips:
	https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=mips&ver=0.9.2-1&stamp=1307570195
	powerpc:
	https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=powerpc&ver=0.9.2-1&stamp=1307550913
	s390:
	https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=s390&ver=0.9.2-1&stamp=1307641748
	sparc:
	https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=sparc&ver=0.9.2-1&stamp=1307552926

2011-06-14  Osier Yang  

	util: Cleanup indention problem in virterror.c
	Push under trivial rule.

2011-06-14  Osier Yang  

	qemu: Parse current balloon value returned by query_balloon
	Qemu once supported following memory stats which will returned by
	"query_balloon":

	    stat_put(dict, "actual", actual);
	    stat_put(dict, "mem_swapped_in", dev->stats[VIRTIO_BALLOON_S_SWAP_IN]);
	    stat_put(dict, "mem_swapped_out", dev->stats[VIRTIO_BALLOON_S_SWAP_OUT]);
	    stat_put(dict, "major_page_faults", dev->stats[VIRTIO_BALLOON_S_MAJFLT]);
	    stat_put(dict, "minor_page_faults", dev->stats[VIRTIO_BALLOON_S_MINFLT]);
	    stat_put(dict, "free_mem", dev->stats[VIRTIO_BALLOON_S_MEMFREE]);
	    stat_put(dict, "total_mem", dev->stats[VIRTIO_BALLOON_S_MEMTOT]);

	But it later disabled all the stats except "actual" by commit
	07b0403dfc2b2ac179ae5b48105096cc2d03375a.

	libvirt doesn't parse "actual", so user will always see a empty result
	with "virsh dommemstat $domain". Even qemu haven't disabled the stats,
	we should support parsing "actual".

2011-06-14  Taku Izumi  

	vcpupin: Fix cpu affinity setting bug of qemu driver
	There is the case where cpu affinites for vcpu of qemu doesn't work
	correctly. For example, if only one vcpupin setting entry is provided
	and its setting is not for vcpu0, it doesn't work.

	   # virsh dumpxml VM
	   ...
	   4
	   
	     
	   
	   ...

	   # virsh start VM
	   Domain VM started

	   # virsh vcpuinfo VM
	   VCPU:           0
	   CPU:            31
	   State:          running
	   CPU time:       2.5s
	   CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

	   VCPU:           1
	   CPU:            12
	   State:          running
	   CPU time:       0.9s
	   CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

	   VCPU:           2
	   CPU:            30
	   State:          running
	   CPU time:       1.5s
	   CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

	   VCPU:           3
	   CPU:            13
	   State:          running
	   CPU time:       1.7s
	   CPU Affinity:   yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

	This patch fixes this problem.

2011-06-13  Matthias Bolte  

	Fix dlopen dependency
	Since the addition of the lock manager framework in 6a943419c528fdd7
	dlopen is always required, but the checks in configure wasn't changed
	to reflect that. This didn't show up directly because the VirtualBox
	driver linking dlopen in covered it. But disabling the VirtualBox
	driver makes the build fail due to missing dlopen.

	Change the dlopen check in configure to pick up dlopen when available.

	Reported by Ruben Kerkhof.

2011-06-13  Hu Tao  

	Deprecate several CURRENT/LIVE/CONFIG enums
	This patch deprecates following enums:

	VIR_DOMAIN_MEM_CURRENT
	VIR_DOMAIN_MEM_LIVE
	VIR_DOMAIN_MEM_CONFIG

	VIR_DOMAIN_VCPU_LIVE
	VIR_DOMAIN_VCPU_CONFIG

	VIR_DOMAIN_DEVICE_MODIFY_CURRENT
	VIR_DOMAIN_DEVICE_MODIFY_LIVE
	VIR_DOMAIN_DEVICE_MODIFY_CONFIG

	And modify internal codes to use virDomainModificationImpact.

2011-06-13  Stefan Berger  

	qemu: Faster response time to qemu startup errors
	The below patch decreases the response time of libvirt to errors reported by Qemu upon startup by checking whether the qemu process is still alive while polling for the local socket to show up.

	This patch also introduces a special handling of signal for the Win32 part of virKillProcess.

2011-06-13  Eric Blake  

	build: update to latest gnulib
	* .gnulib: Update to latest, for more strerror_r fixes.

2011-06-13  Taku Izumi  

	vcpupin: add the new option to "virsh vcpupin" command
	This patch adds the new option (--live, --config and --current) to
	"virsh vcpupin" command. The behavior of above aption is the same as
	that of "virsh setmem", "virsh setvcpus", and whatnot.
	When the --config option is specified, the command affects a persistent
	domain, while --live option is specified, it affects a running (live) domain.
	The --current option cannot be used with --config or --live at the same
	time, and when --current is specified, it affects a "current" domain.

	vcpupin: implement the remote protocol to address the new API
	This patch implements the remote protocol for the new API
	(virDomainPinVcpuFlags).

	vcpupin: implement the code to address the new API in the qemu driver
	This patch implements the new API (virDomainPinVcpuFlags) in the qemu
	driver.

	vcpupin: introduce a new libvirt API (virDomainPinVcpuFlags)
	This patch introduces a new libvirt API virDomainPinVcpuFlags,
	a direct extension from the existing virDomainPinVcpu

2011-06-13  Jiri Denemark  

	Use virTimeMs when appropriate

	Introduce virTimeMs for getting current time in ms

	test: Remove unused timeval

2011-06-13  Wen Congyang  

	tests: add a test for multi function PCI device

2011-06-13  Wen Congyang  

	support multifunction PCI device
	If qemu supports multi function PCI device, the format of the PCI address passed
	to qemu is "bus=pci.0,multifunction=on,addr=slot.function".

	If qemu does not support multi function PCI device, the format of the PCI address
	passed to qemu is "bus=pci.0,addr=slot".

2011-06-13  Wen Congyang  

	the hotplugged PCI device should use the whole slot
	Hot pluging/unpluging multi PCI device is not supported now. So the function
	of hotplugged PCI device must be 0. When we hot unplug it, we should set release
	all functions in the slot.

	assign the whole slot to the PCI device that has no address
	If user does not specify the PCI address, we should auto assign an unused slot.

	Reimplement qemuDomainPCIAddressReserveSlot(): reserve all functions in the slot
	We will support multi function PCI device. So we should reserve all functions in
	the slot if we want to reserve a slot.

	the key of hash table should include the function value
	We save all used PCI address in the hash table. The key is generated by domain,
	bus and slot now. We will support multi function PCI device, so the key should
	be generated by domain, bus, slot and function.

	prevent hot unplugging multi function PCI device
	We do not support to hot unplug multi function PCI device now. If the device is
	one function of multi function PCI device, we shoul not allow to hot unplugg
	it.

	check whether qemu supports multi function PCI device
	qemu supports multi function PCI device at least version 0.13.0.

2011-06-10  Matthew Booth  

	xenapi: Improve error message on session failure
	XenAPI session login can fail for a number of reasons, but currently no
	specific
	reason is displayed to the user, e.g.:

	virsh -c XenAPI://citrix-xen.example.com/
	Enter username for citrix-xen.example.com: root
	Enter root's password for citrix-xen.example.com:
	error: authentication failed: (null)
	error: failed to connect to the hypervisor

	This patch displays the session error description on failure.

2011-06-10  Osier Yang  

	qemu: Fix one type in the error prompt string
	s/hostdevwork/hostdev/

2011-06-09  Matthias Bolte  

	Move VMware Workstation/Player driver to correct spec file section
	The VMware driver works like the OpenVZ driver by using a commandline
	tool for management. It dosen't use it's own remote protocol.

2011-06-08  Eric Blake  

	virt-aa-helper: add missing include
	Regression introduced in commit 02e8691.

	* src/security/virt-aa-helper.c (includes): Reflect move of virRun.

2011-06-08  Eric Blake  

	daemon: plug memory leak
	Detected by Coverity.  Commit ef21beda was incomplete; it solved
	a leak one one path, but not on the other.

	* daemon/libvirtd.c (qemudSetLogging): Avoid leak on success.

2011-06-08  Eric Blake  

	build: break some long lines
	As long as I was already touching the function...

	* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphics): Line wrap.

2011-06-08  Eric Blake  

	qemu: add missing break statement
	Detected by Coverity.  Bug introduced in commit 9d73efd (v0.8.8).

	* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphics): Don't report
	error on success.

2011-06-08  Eric Blake  

	build: silence coverity false positives
	Coverity complained about these intentional fallthrough cases, but
	not about other cases that were explicitly marked with nice comments.

	For some reason, Coverity doesn't seem smart enough to parse the
	up-front English comment in virsh about intentional fallthrough :)

	* tools/virsh.c (cmdVolSize): Mark fallthrough in a more typical
	fashion.
	* src/conf/nwfilter_conf.c (virNWFilterRuleDefDetailsFormat)
	(virNWFilterRuleDetailsParse): Mark explicit fallthrough.

2011-06-08  Eric Blake  

	esx: avoid dead code
	Detected by Coverity.  The beginning of the function already filtered
	out NULL objectContentList as invalid.  Further investigation shows:

	esxVI_RetrieveProperties is generated and returns a list of objects
	that match the given propertyFilterSpec.
	esxVI_LookupObjectContentByType then tests whether the result
	corresponds to the expected occurrence and reports an error otherwise.
	This simplifies the callers of  esxVI_LookupObjectContentByType, but
	due to the missing dereference the check was never performed because
	the code thought that at least one item was obtained. NULL represents
	an empty list. This is a potential segfault fix because callers of
	esxVI_LookupObjectContentByType that specified "required" occurrence
	assume *objectContentList to be non-NULL when
	esxVI_LookupObjectContentByType succeeds.

	* src/esx/esx_vi.c (esxVI_LookupObjectContentByType): Check
	correct pointer.

2011-06-08  Eric Blake  

	secret: drop dead code
	Detected by Coverity.  The only ways to get to the cleanup label
	were by an early abort (list still unassigned) or after successfully
	transferring list to dest, so there is no list to clean up.

	* src/secret/secret_driver.c (loadSecrets): Kill dead code.

2011-06-08  Eric Blake  

	qemu: reorder checks for safety
	Detected by Coverity.  All existing callers happen to be in
	range, so this isn't too serious.

	* src/qemu/qemu_cgroup.c (qemuCgroupControllerActive): Check
	bounds before dereference.

2011-06-08  Eric Blake  

	uuid: annotate non-null requirements
	Coverity already saw through a NULL dereference without these
	annotations, and gcc is still too puny to do good NULL analysis.
	But clang still benefits (and is easier to run than coverity),
	not to mention that adding this bit of documentation to the code
	may help future developers remember the constraints.

	* src/util/uuid.h (virGetHostUUID, virUUIDFormat): Document
	restrictions, for improved static analysis.

2011-06-08  Eric Blake  

	debug: avoid null dereference on uuid lookup api
	Detected by Coverity.  Commit a98d8f0d tried to make uuid debugging
	more robust, but missed some APIs.  And on the APIs that it visited,
	the mere act of preparing the debug message ends up dereferencing
	uuid prior to the null check.  Which means the APIs which are supposed
	to gracefully reject NULL arguments now end up with SIGSEGV.

	* src/libvirt.c (VIR_UUID_DEBUG): New macro.
	(virDomainLookupByUUID, virDomainLookupByUUIDString)
	(virNetworkLookupByUUID, virNetworkLookupByUUIDString)
	(virStoragePoolLookupByUUID, virStoragePoolLookupByUUIDString)
	(virSecretLookupByUUID, virSecretLookupByUUIDString)
	(virNWFilterLookupByUUID, virNWFilterLookupByUUIDString): Avoid
	null dereference.

2011-06-08  Eric Blake  

	python: avoid unlikely sign extension bug
	Detected by Coverity.  cpumap was allocated with a value of
	(unsigned short)*(int), which is an int computation, and then
	promotes to size_t.  On a 64-bit platform, this fails if bit
	32 of the product is set (because of sign extension giving
	a HUGE value to malloc), even though a naive programmer would
	assume that since the first value is unsigned, the product
	is also unsigned and at most 4GB would be allocated.

	Won't bite in practice (the product should never be that large),
	but worth using the right types to begin with, so that we are
	now computing (unsigned short)*(size_t).

	* python/libvirt-override.c (libvirt_virDomainGetVcpus): Use
	correct type.

2011-06-08  Eric Blake  

	build: silence coverity false positive
	Similar in nature to commit fd21ecfd, which shut up valgrind.

	sigaction is apparently a nasty interface for analyzer tools,
	at least for how many false positives it generates.

	* src/util/command.c (virExecWithHook): Initialize entire var, since
	coverity gripes about the (unused and non-standard) sa_restorer.

2011-06-08  Eric Blake  

	storage: avoid mishandling backing store > 2GB
	Detected by Coverity.  The code was doing math on shifted unsigned
	char (which promotes to int), then promoting that to unsigned long
	during assignment to size.  On 64-bit platforms, this risks sign
	extending values of size > 2GiB.  Bug present since commit
	489fd3 (v0.6.0).

	I'm not sure if a specially-crafted bogus qcow2 image could
	exploit this, although it's probably not possible, since we
	were already checking for the computed results being within
	range of our fixed-size buffer.

	* src/util/storage_file.c (qcowXGetBackingStore): Avoid sign
	extension.

2011-06-08  Eric Blake  

	build: detect Coverity 5.3.0
	Coverity 5.3.0 still outputs lots of COVERITY_* variables, but no
	longer modifies COVERITY_BUILD_COMMAND in the environment.  Pick
	one that seems likely to stay around.

	* configure.ac (STATIC_ANALYSIS): Detect newer Coverity.

2011-06-08  Osier Yang  

	build: Fix typos in configure.ac

	virsh: Expose virDomainMigrateSetMaxSpeed API to virsh
	API virDomainMigrateSetMaxSpeed was introduced since 0.9.0, but
	no command in virsh yet.

2011-06-07  Cole Robinson  

	lxc: Ensure container  actually exists
	Since we can't really get useful error reporting from virCommandExec since
	it needs to be the last thing we do.

	lxc: Verify root fs exists before mounting
	Otherwise the following virFileMakePath will create the directory for
	us and fail further ahead, which probably isn't intended.

	lxc: controller: Improve container error reporting
	Add a handshake with the cloned container process to try and detect
	if it fails to start.

	lxc: Improve guest startup error reporting
	Add a simple handshake with the lxc_controller process so we can detect
	process startup failures. We do this by adding a new --handshake cli arg
	to lxc_controller for passing a file descriptor. If the process fails to
	launch, we scrape all output from the logfile and report it to the user.

	lxc: Refactor controller command building
	Arranges things similar to the qemu driver. Will allow us to more easily
	report command error output.

	lxc: Don't report error in Wait/SendContinue
	We will reuse these shortly, and each use should have a different error
	message.

	lxc: Drop container stdio as late as possible
	Makes it more likely we get useful error output in the logs

2011-06-07  Cole Robinson  

	Move virRun, virExec*, virFork to util/command
	Seems reasonable to have all command wrappers in the same place

	v2:
	    Dont move SetInherit

	v3:
	    Comment spelling fix
	    Adjust WARN0 comment
	    Remove spurious #include movement
	    Don't include sys/types.h
	    Combine virExec enums

2011-06-07  Cole Robinson  

	util: Implement virRun as a wrapper around virCommand
	v2:
	    Simplify command building
	    Handle command building failure

	v3:
	    Remove unneeded NULL check

2011-06-07  Cole Robinson  

	util: Remove unused virExec wrapper
	v3:
	    Remove obsolete comment

2011-06-07  Cole Robinson  

	qemu: Convert virExec usage to virCommand
	v2:
	    Have virCommand cleanup intermediate process for us

	v3:
	    Preserve original FD closing behavior

2011-06-07  Matthias Bolte  

	esx: Remove duplicated invalid-argument checks
	Those checks are already performed at the public API level.

2011-06-07  Osier Yang  

	docs: Add doc for video element
	For backwards compatibility, if no