# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. Bigtop Changelog * Release 1.2.0 (2017-04-04) Release Notes - Bigtop - Version 1.2.0 ** Sub-task * [BIGTOP-1406] - package Ambari in Bigtop * [BIGTOP-1408] - create basic end-to-end tests for Ambari integration * [BIGTOP-1409] - consider using ambari shell * [BIGTOP-1624] - Add puppet recipes for deploying kafka * [BIGTOP-2179] - Apache Tajo to bigtop: packaging as deb/rpm * [BIGTOP-2180] - Apache Tajo to bigtop: make tests * [BIGTOP-2224] - Let's bump groovy version for smokes to 2.4.10 * [BIGTOP-2254] - Replace the Docker orchestration tool from Vagrant to Docker Compose * [BIGTOP-2285] - Add qfs rpm and debian packaging code * [BIGTOP-2293] - Add puppet recipes for qfs components * [BIGTOP-2312] - Add environment check * [BIGTOP-2314] - Added deb and rpm package of Apache Apex to bigtop. * [BIGTOP-2315] - Add smoke test for apex * [BIGTOP-2316] - Add apex puppet receipes * [BIGTOP-2317] - Add smoke tests for QFS * [BIGTOP-2319] - Build initial smoke-tests distribution * [BIGTOP-2326] - Build salves need to have the toolchain run again to install qfs dependencies * [BIGTOP-2345] - Create Flink packaging * [BIGTOP-2357] - Create puppet recipes * [BIGTOP-2468] - Add Juju hadoop-processing bundle * [BIGTOP-2469] - Add cloud-weather-report test plan * [BIGTOP-2491] - Update provisioner/docker-hadoop.sh to bind with provisioner/utils * [BIGTOP-2492] - Split flink debian packaging * [BIGTOP-2505] - Support systemd containers, clean up hiera.yaml handling, fix exec (-e) flag * [BIGTOP-2518] - Add node to the build slave configuration * [BIGTOP-2526] - Bump flink version to 1.1.3 * [BIGTOP-2530] - Create Greenplum packages * [BIGTOP-2531] - Create Greenplum deployment scripts * [BIGTOP-2532] - Create Greenplum test * [BIGTOP-2600] - Bump Groovy version to 2.4.10 * [BIGTOP-2601] - Bump Ignite to 1.9 * [BIGTOP-2613] - create bigtop/puppet docker image or fedora 25 * [BIGTOP-2614] - create bigtop/slave docker image for fedora 25 * [BIGTOP-2620] - Bump oozie version to 4.3.0 * [BIGTOP-2624] - Bump Phoenix version to 4.9.0 * [BIGTOP-2625] - update crunch for JAVA 8 * [BIGTOP-2626] - HBase build fails when JAVA 8 is used * [BIGTOP-2627] - Kite 1.1.0 build fails when JAVA 8 is used * [BIGTOP-2646] - Move Vagrant Provisioner from bigtop-deploy/vm to provisioner directory * [BIGTOP-2647] - Clean up code under bigtop-deploy/vm after everything moved to provisioner directory * [BIGTOP-2657] - Upgrade to Hue 3.11.0 * [BIGTOP-2668] - Polish and update Docker Provisioner configurations * [BIGTOP-2672] - Update gradle wrapper for Docker Provisioner * [BIGTOP-2700] - Adding tests to test the doas feature of httpfs * [BIGTOP-2701] - Update hcfs tests so it can work in both casers where fs.trash.interval is disabled or enabled. * [BIGTOP-2702] - Fix Sandbox creation script * [BIGTOP-2704] - Include ODPi runtime tests option into the battery of smoke tests * [BIGTOP-2705] - provide puppet deployment code for Ambari ** Bug * [BIGTOP-976] - package deployment tests are CDH specific: _has_ to be removed or fixed * [BIGTOP-1533] - failed to load/initialize native-bzip2 library system-native * [BIGTOP-2047] - detect JAVA_HOME for JDK8, remove obsolete detection code (JDK6/oracle JDK) * [BIGTOP-2133] - Running BPSGenerator using fat-jar fails on Mac OS X * [BIGTOP-2134] - Wrong package name in bigpetstore-mapreduce/arch.dot * [BIGTOP-2135] - PigCSVCleaner fails due to the lack of the dependent jar * [BIGTOP-2136] - A comment about parameter substitution in BPS_analytics.pig is slightly wrong * [BIGTOP-2138] - deb: make bigtop-jsvc a wrapper package for jsvc package * [BIGTOP-2220] - flume-agent.init incorrectly handles flume.conf * [BIGTOP-2225] - Bump toolchain gradle to 2.10 * [BIGTOP-2229] - bigtop deploy to support centos-7 * [BIGTOP-2231] - build.gradle carries one too many sets of repositories * [BIGTOP-2261] - adding bigtop/puppet:fedora-22 build support for ppc64le * [BIGTOP-2274] - CLONE - rpm: need to make bigtop-jsvc a wrapper package for jsvc package * [BIGTOP-2301] - Bigtop Homepage shows wrong url to CI * [BIGTOP-2302] - Use apt instead of yum in setup-env-debian.sh * [BIGTOP-2303] - Fix the indentation in docker-hadoop.sh * [BIGTOP-2308] - Clean up build directory after sucessfull build of package * [BIGTOP-2318] - Release assembly needs to be updated * [BIGTOP-2340] - BIGTOP-2319 is incomplete: the code for smoke-tests is missing * [BIGTOP-2342] - Set yarn.log.server.url to point to JH server * [BIGTOP-2346] - Do not use gradle delete() for sources and build directories * [BIGTOP-2347] - Clean up build directory after sucessfull build of package (2nd try) * [BIGTOP-2350] - HCatalog WebHCat server's default file incorrectly specifies HCAT_PREFIX AND HADOOP_PREFIX * [BIGTOP-2353] - fix qfs suse build * [BIGTOP-2354] - qfs does not build on fedora * [BIGTOP-2358] - Update the URL for Bigtop-trunk-packages job on README * [BIGTOP-2359] - Add .DS_Store to .gitignore * [BIGTOP-2362] - Enabling bigtop docker provisioner for ppc64le (ubuntu 15.04) * [BIGTOP-2366] - Support https:// repositories for DEB systems * [BIGTOP-2367] - QFS packages do not handle /var/run/qfs properly * [BIGTOP-2372] - Puppet deploy README points to non-existing repo * [BIGTOP-2373] - kinit path is not correctly configured in hue.ini template for CentOs by puppet module * [BIGTOP-2374] - Toolchain needs clear warning for Puppet <3 * [BIGTOP-2376] - Update Tomcat and remove obsolete downloads * [BIGTOP-2377] - Update to Hadoop 2.7.2 * [BIGTOP-2378] - Do not use archive.apache.org for ant download * [BIGTOP-2379] - update maven to 3.3.9 and refactor toolchain * [BIGTOP-2380] - support opensuse-42.1 (leap) * [BIGTOP-2382] - Parameterise and expose "default_hdfs_superuser" for hue.ini * [BIGTOP-2399] - Maven artifacts are still on 2.6.0 version of Apache Hadoop * [BIGTOP-2400] - Get groovy plugins used by Maven in sync with Bigtop groovy runtime * [BIGTOP-2401] - a commiter name is missing in the "Who we are" page * [BIGTOP-2402] - yet another commiter name is missing in the "Who we are" page * [BIGTOP-2404] - puppetize.sh fails on Ubuntu 14.04 because of the lack of apt-get update * [BIGTOP-2405] - Rollback BIGTOP-2049 as the original issue was fixed in the upstream * [BIGTOP-2406] - init scripts do not work reliable on systemd systems * [BIGTOP-2412] - arm dockerfile breaks RAT * [BIGTOP-2413] - QFS README.md breaks RAT * [BIGTOP-2419] - Fix symlinks on packaging Crunch rpm and deb * [BIGTOP-2421] - Ignite RPM is broken: "missing" osgi package deps * [BIGTOP-2423] - Fix hard-coded port number in Sqoop smokes * [BIGTOP-2425] - crunch needs JDK8 javadoc * [BIGTOP-2427] - jsvc build not recognizing ARM as valid platform * [BIGTOP-2430] - Adding Dockerfile to build puppet image for ubuntu-16.04-ppc64le * [BIGTOP-2431] - Adding Dockerfile to build slaves image for ubuntu-16.04-ppc64le * [BIGTOP-2433] - mvn is missing in Bigtop Toolchain * [BIGTOP-2434] - Change Spark test to unpack jar file and retrieve resources * [BIGTOP-2439] - flink: fix adduser and register init.d scripts on DEB systemes * [BIGTOP-2441] - spark smoke test doesn't work relying on yarn-client submittion * [BIGTOP-2445] - Adding Dockerfile to build deploy image for ubuntu-16.04-ppc64le * [BIGTOP-2448] - Adding Dockerfile to build puppet image for ubuntu-16.04-arm * [BIGTOP-2449] - Adding Dockerfile to build slaves image for ubuntu-16.04-x86 * [BIGTOP-2450] - Adding Dockerfile to build slaves image for ubuntu-16.04-arm * [BIGTOP-2460] - docker-hadoop.sh fails when deploying on ubunt 16.04 images * [BIGTOP-2463] - Regression: bigtop-jsvc build fails on ppc64le architecture * [BIGTOP-2464] - Outdated descriptions in vagrant-puppet-vm/README.md * [BIGTOP-2471] - qfs does not build on opensuse:42.1 * [BIGTOP-2472] - zeppelin puppet module needs to expose ZEPPELIN_WAR_TEMPDIR * [BIGTOP-2473] - Mahout puppet script fails to find Package[hadoop] * [BIGTOP-2474] - Improve handling of gradle home * [BIGTOP-2487] - Alluxio /var/log/alluxio is not created * [BIGTOP-2488] - Redundant zookeeper/lib directory * [BIGTOP-2494] - Use systemd within bigtop/deploy docker image for debian-8, centos-7, fix centos-7 provisioner * [BIGTOP-2495] - hive-hbase package is broken * [BIGTOP-2509] - Path error in bigtop-packages/src/common/solr/tomcat-deployment.sh * [BIGTOP-2512] - Wrong help message in docker-hadoop.sh * [BIGTOP-2513] - Some RAT exclude rules defined in build.gradle are not defined in pom.xml * [BIGTOP-2523] - Incorrect cycle dependency during puppet deployment causes error at Debian 8 * [BIGTOP-2527] - Recreate all bigtop docker images and install "host" on debian * [BIGTOP-2528] - provisioner fails to add short host name * [BIGTOP-2529] - Vagrant Provisioner failed to start up daemons * [BIGTOP-2533] - Use TLS for downloading sources * [BIGTOP-2535] - frontend-maven-plugin fails on ppc64le * [BIGTOP-2536] - tarball.destination is ignored when set * [BIGTOP-2537] - Hive smoke test identifies as org.apache.bigtop.itest.hadoop.mapreduce * [BIGTOP-2542] - Some files related to docker provisioner breaks RAT * [BIGTOP-2543] - Some minor problems on provisioner/docker/docker-hadoop.sh * [BIGTOP-2545] - Add ZOOKEEPER-2594 to bigtop * [BIGTOP-2546] - Make debian/ubuntu toolchain install more resistent agains failures * [BIGTOP-2549] - "Expected" and "actual" values are mistakenly switched in some JUnit method invocation * [BIGTOP-2551] - docker-hadoop.sh --create fails with "Error: Could not match |$index" message * [BIGTOP-2553] - namenode ports are not configured for non-HA mode * [BIGTOP-2556] - Building Hadoop fails because Tomcat 6.0.45 is not downloadable from Apache mirror * [BIGTOP-2557] - BIGTOP-2536 broke DataFu, QFS and YCSB * [BIGTOP-2558] - Add ppc64le and arm64 node to the build slave configuration * [BIGTOP-2559] - Revert JIRA BIGTOP-2535 * [BIGTOP-2560] - Spark charm failing automated tests * [BIGTOP-2563] - zeppelin build fails on ppc64le * [BIGTOP-2564] - HBase build fails on Power * [BIGTOP-2565] - upgrade Spark to version 1.6.2 * [BIGTOP-2566] - Specify KAFKA_VERSION in build command for Flume * [BIGTOP-2569] - Spark 2.0 * [BIGTOP-2570] - ease hadoop charm debugging * [BIGTOP-2582] - Adding Spark1 components for Spark 1.x family * [BIGTOP-2585] - Zookeeper service does not start if kerberos is disabled * [BIGTOP-2587] - vagrant-puppet-vm fails with "Could not find class node_with_components" * [BIGTOP-2588] - Spark 2.0.1 installation fails on DEB * [BIGTOP-2589] - spark1 build break * [BIGTOP-2590] - Flume build break on RPM * [BIGTOP-2591] - Fix bug in BIGTOP-2569 * [BIGTOP-2593] - Build fail caused by tomcat 6.0.45 version * [BIGTOP-2594] - Workaround to fix Hive Build, by using SPARK1 Version * [BIGTOP-2596] - Phoenix build fails * [BIGTOP-2597] - crunch build failed * [BIGTOP-2599] - Zeppelin 0.6.2 build fails on Power * [BIGTOP-2602] - ignite-hadoop build break * [BIGTOP-2603] - NN/RM charm should include a spark user/group * [BIGTOP-2604] - Build flink-dist with proper shading * [BIGTOP-2605] - Addendum for BIGTOP-2514 * [BIGTOP-2606] - spark1 build fails when RPM is sellected * [BIGTOP-2608] - upgrade Hadoop to 2.7.3 * [BIGTOP-2609] - upgrade Kafka to version 0.10.0.0 * [BIGTOP-2610] - upgrade mahout to version 0.12.2 * [BIGTOP-2612] - Need bigtop/puppet image for Fedora-25 * [BIGTOP-2618] - bigtop-jsvc build fails with OpenJDK-1.8 * [BIGTOP-2621] - JAVA_HOME is not set on Fedora-25+JAVA-1.8 for x86 * [BIGTOP-2623] - Zeppelin 0.6.2 build fails on all platforms * [BIGTOP-2628] - sqoop2 build fails when JAVA 8 is used * [BIGTOP-2632] - bigtop 1.1.0 sqoop2 wrote rpm error * [BIGTOP-2635] - ubuntu on ppc64le is missing a apt-get update * [BIGTOP-2636] - Fix giraph after upgrade to Hadoop-2.7.3 * [BIGTOP-2637] - Fix flume because of kafka 0.10 update * [BIGTOP-2638] - Update to kafka 0.10.1.0 * [BIGTOP-2640] - Incomplete patch BIGTOP-2635 - Bigtop stack build fails on Ubuntu 16.04-ppc64le * [BIGTOP-2643] - Force giraph to compile by removing findbugs from all pom.xml * [BIGTOP-2644] - CI compiles random greenplum source * [BIGTOP-2648] - Gradle 2.7 SSL Problems * [BIGTOP-2649] - Default Java8 blocks default debian jdk * [BIGTOP-2650] - Docker build should pull up to date packages * [BIGTOP-2651] - Delete obsolete docker images and build files. * [BIGTOP-2652] - tajo rpm packaging seriously broken * [BIGTOP-2653] - upgrading to Spark 2.1 * [BIGTOP-2654] - spark 2.1 binaries need either SPARK_HOME or non existing find-spark-home exe * [BIGTOP-2655] - Help spark find the native hadoop libs * [BIGTOP-2656] - regression - spark 2.1 upgrade breaks rpm packaging * [BIGTOP-2661] - Docker deploy images should pull upstream images * [BIGTOP-2662] - provisioner: Add support for JDK8 on debian * [BIGTOP-2663] - puppet hadoop module: Consolidate memory resource settings * [BIGTOP-2664] - create provisioner test for trunk packages with jdk8 * [BIGTOP-2665] - Fix hue for opensuse * [BIGTOP-2669] - Create bigtop-1.2 repository * [BIGTOP-2673] - Need to back port HADOOP-11628 for JDK8 * [BIGTOP-2674] - docker-hadoop option -C does not work correctly any more * [BIGTOP-2675] - [Puppet] Kafka can only be deployed on master node * [BIGTOP-2676] - Smoke test fails on Spark 2 * [BIGTOP-2678] - Add hive-hbase role to install hive-hbase rpm * [BIGTOP-2689] - Upgrade Zeppelin to version 0.7 * [BIGTOP-2690] - gradlew toolchain fails trying to download Ant 1.9.8 * [BIGTOP-2692] - Flume init script should load only '.conf' extension files * [BIGTOP-2693] - Update readme for build and configuring git repo for packages * [BIGTOP-2695] - arch.dot for bigpetstore-spark is outdated * [BIGTOP-2699] - Zeppeling CI build is broken * [BIGTOP-2707] - localhost is missing in docker provisioner instances * [BIGTOP-2708] - puppet code always installs hdfs by default * [BIGTOP-2710] - qfs debian build is breaking because it is confused about Maven home * [BIGTOP-2711] - fix rat problems * [BIGTOP-2715] - Tiny typo in the README.md * [BIGTOP-2718] - Mark tez package as arch dependent * [BIGTOP-2719] - Mark zeppelin package as arch dependent * [BIGTOP-2721] - libsnappy reference path is not correct for hadoop/ubuntu16 build * [BIGTOP-2724] - putting a final nail in Java's 7 coffin * [BIGTOP-2725] - a few usability improvements to run_itest.sh * [BIGTOP-2726] - Bump ambari version to 2.5 * [BIGTOP-2727] - fix deployment tasks ** Improvement * [BIGTOP-1608] - Create Unified testing solution: Smoke-Tests and Test-Artifacts * [BIGTOP-1843] - Upgrade Sqoop to 1.4.6 * [BIGTOP-2077] - Bump HBase version to 1.1 * [BIGTOP-2118] - Update HBase to 0.98.17 * [BIGTOP-2306] - Add support for specifying git repository access credentials * [BIGTOP-2309] - Update submodules after git clone * [BIGTOP-2327] - Bump up Phoenix to 4.7.0 * [BIGTOP-2330] - Bump Tachyon version to 0.8.2 * [BIGTOP-2332] - Upgrade Tajo version to 0.11.1 * [BIGTOP-2333] - Make Apache Pig work with HBase 1.1 on Hadoop 2 * [BIGTOP-2337] - Fix script to make deb in Tajo * [BIGTOP-2351] - Vagrant VM is missing /user/vagrant HDFS directory * [BIGTOP-2352] - Packaged Vagrant box has incorrect host info on restart * [BIGTOP-2370] - Upgrade DataFu version to 1.3.0 * [BIGTOP-2381] - Making Zookeeper hosts templatize in puppet module of hue * [BIGTOP-2384] - Refactor toolchain code to extract packages version out of every action * [BIGTOP-2385] - Reformat bigtop_toolchain/manifests/packages.pp to ease maintenance * [BIGTOP-2393] - Add ARM64 support for build environment * [BIGTOP-2395] - Create separate page to point "Download now" link to * [BIGTOP-2403] - Remove affiliation column in the team list * [BIGTOP-2407] - Making "force_username_lowercase" variable templatize in puppet module of hue * [BIGTOP-2408] - group_filter variable value is not templatize in puppet module of hue * [BIGTOP-2409] - nt_domain configuration option for hue to connect to Active Directory * [BIGTOP-2410] - Making "ldap_username_pattern" variable templatize in puppet module of hue * [BIGTOP-2411] - Add OS-neutral wrappers to build all native packages and repositories * [BIGTOP-2416] - Upgrade Crunch to 0.13.0 * [BIGTOP-2429] - Add Solr to gradle smokes * [BIGTOP-2443] - inject unzip method to ZipInputStream, accepting regex for include * [BIGTOP-2455] - support pre-installed java environment * [BIGTOP-2458] - Add option to disable IP hostname checking for DataNode registration * [BIGTOP-2459] - Add option to disable vmem check for NodeManager * [BIGTOP-2466] - HBase puppet scripts co-deploy ZK and Datanode along with HBase * [BIGTOP-2478] - Zookeeper does not have any smoke tests * [BIGTOP-2479] - Update qfs maintainer to kstinson * [BIGTOP-2490] - Spark in HA when Zookeeper is available * [BIGTOP-2504] - Kafka should be able to bind to something other than 0.0.0.0/the default interface * [BIGTOP-2506] - Zookeeper: non default interface for client * [BIGTOP-2507] - Make buildstamp configurable by a bom file * [BIGTOP-2516] - Add Zeppelin Charm * [BIGTOP-2521] - Problem with deprecated methods in puppet code with puppet option parser=future * [BIGTOP-2522] - Add spark processing juju bundle * [BIGTOP-2547] - Don't render ip-hostname-check in hdfs-site.xml if default * [BIGTOP-2548] - Refresh charms for Juju 2.0 and Xenial * [BIGTOP-2554] - expose bind-host options in hieradata * [BIGTOP-2555] - hadoop charms should use bind-host overrides * [BIGTOP-2561] - add juju bundle for hadoop-spark * [BIGTOP-2571] - cwr driven updates to the zeppelin juju charm * [BIGTOP-2575] - zk charm test updates * [BIGTOP-2576] - For small clusters it is useful to turn replace-datanode-on-failure off * [BIGTOP-2577] - kafka charm test updates * [BIGTOP-2578] - Puppet needs to be updated for Hive Server2 * [BIGTOP-2579] - Puppet needs to be updated for Hive Metastore service * [BIGTOP-2611] - Adding Fedora-25 with Java 1.8 support * [BIGTOP-2616] - refresh juju hadoop-processing bundle * [BIGTOP-2617] - refresh juju spark-processing bundle * [BIGTOP-2682] - update juju bundles with recent charm revs * [BIGTOP-2688] - Add flag for provisioning of Yum Repo * [BIGTOP-2694] - Added Hernan Vivani (hvivani) as elasticsearch maintainer * [BIGTOP-2696] - Build out VM for sandboxes * [BIGTOP-2697] - Back port HADOOP-12366 into Bigtop * [BIGTOP-2712] - Juju CI driven updates * [BIGTOP-2714] - Update Groovy in the toolchain to 2.4.10 ** New Feature * [BIGTOP-1641] - Add packaging for Apache Tajo * [BIGTOP-2282] - Define Bigtop 1.2 release BOM * [BIGTOP-2296] - Provide a way to build Docker container with functional stack * [BIGTOP-2339] - add centos-7 to the provisioiner matrix * [BIGTOP-2435] - Add Juju charms for hadoop component * [BIGTOP-2451] - Adding Ubuntu 16.04 support to Bigtop * [BIGTOP-2452] - Adding Dockerfile to build deploy image for ubuntu-16.04-x86 * [BIGTOP-2476] - Add Zookeeper Charm * [BIGTOP-2477] - Add Juju charm for spark component * [BIGTOP-2481] - Add HBase Charm * [BIGTOP-2482] - Adding Pig charm * [BIGTOP-2483] - Add Mahout Charm * [BIGTOP-2486] - Add Kafka Charm * [BIGTOP-2524] - Add Greenplum component * [BIGTOP-2615] - Provide a tool to build pseudo cluster docker images * [BIGTOP-2660] - Add Giraph Charm ** Task * [BIGTOP-2289] - Set master version to 1.2.0-SNAPSHOT * [BIGTOP-2334] - Update latest release link on the website * [BIGTOP-2335] - ci link should use https:// authority * [BIGTOP-2368] - OpenStack config bigtop repo-url changes. * [BIGTOP-2371] - Add Oozie to gradle smoke tests * [BIGTOP-2414] - Rename Tachyon to Alluxio * [BIGTOP-2415] - readme spelling error * [BIGTOP-2456] - Update Apache Apex version to 3.4.0 * [BIGTOP-2497] - Upgrade Apache Phoenix to 4.8.1 * [BIGTOP-2500] - Remove hadoop jars from TEZ_HOME/lib and create appropriate symlinks * [BIGTOP-2525] - committer test: add myself to team-list.xml * [BIGTOP-2538] - Bump version of Solr to 4.10.4 * [BIGTOP-2550] - Update juju hadoop bundle for Juju 2.0 and Xenial * [BIGTOP-2567] - Upgrade Flume version to 1.7.0 * [BIGTOP-2568] - Upgrade Kafka version to 0.9.0.1 * [BIGTOP-2592] - Upgrade to Spark 2.0.2 * [BIGTOP-2622] - Add Kengo Seki to team list * [BIGTOP-2629] - Add Jonathan Kelly to team list * [BIGTOP-2670] - Bump up Apache Apex version to 3.5.0 * [BIGTOP-2687] - Remove /usr/share/java/*.jar from Sqoop2 catalina.properties * [BIGTOP-2703] - refresh juju charms/bundles with metric and CI support * [BIGTOP-2722] - Remove workaround allowing build by root in Tez ** Test * [BIGTOP-2534] - Flume tests fail with java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets ** Wish * [BIGTOP-2680] - Update kafka to 0.10.1.1 * Release 1.1.0 (2016-01-30) Release Notes - Bigtop - Version 1.1.0 ** Sub-task * [BIGTOP-1494] - Introduce Groovy DSL to replace bigtop.mk * [BIGTOP-1499] - released source code is not same with source code in branch * [BIGTOP-1898] - Dockerfiles to build bigtop/puppet docker images for 1.0 release * [BIGTOP-1899] - Migrate CI master to EC2 instance donated by Amazon EMR team * [BIGTOP-1901] - Provide bigtop/deploy images on dockerhub for CI and users to consume docker provisioner * [BIGTOP-1947] - Fix RAT plugin configuration to be able to RAT-validate all published artifacts * [BIGTOP-1959] - Avoid running top-level assembly while doing deploy for subprojects * [BIGTOP-1965] - Remove the link to 0.8.0 from the website * [BIGTOP-1973] - Add new tests for HDFS Balancer functionality * [BIGTOP-1975] - Smoke tests for Spark SQL * [BIGTOP-1981] - Add new tests for test, text, count commands * [BIGTOP-1983] - Move BigPetStore data generator to bigtop-data-generators * [BIGTOP-1984] - Extract samplers library from BigPetStore data generator * [BIGTOP-1985] - Extract name generator from BigPetStore data generator * [BIGTOP-1986] - Extract location dataset from BigPetStore data generator * [BIGTOP-1990] - Add gradle multi-project build files for bigtop-data-generators * [BIGTOP-1991] - Add BigTop Weatherman * [BIGTOP-1992] - Fix RAT check errors resulting from BPS data generator move * [BIGTOP-1995] - Update BigPetStore to use external locations data * [BIGTOP-2000] - Add BigTop Data Generators to maintainers file * [BIGTOP-2009] - Add new tests for chgrp, cp, ls, mv, du, put, get, mkdir, stat and touchz * [BIGTOP-2025] - Make BOM to be a directional graph * [BIGTOP-2033] - Build order of the stack is broken * [BIGTOP-2051] - Get rid of hair-brain environment vars left after make-based build * [BIGTOP-2055] - Refactor packages.gradle to get rid of excessive data containers; make code cleaner * [BIGTOP-2087] - The 1.0 release package artifacts on Bigtop jenkins is broken due to CI transition * [BIGTOP-2096] - ignite-hadoop service should restart upon changes in the package or configs * [BIGTOP-2097] - cleaning up the ignite-service * [BIGTOP-2104] - Packages upgrade to Spark 1.5.1 * [BIGTOP-2105] - Puppet recipes improvements after Spark is bumped to 1.5.1 * [BIGTOP-2107] - Ignite package still shows website at the incubator * [BIGTOP-2113] - Spark master doesn't bind to the host: ignores SPARK_MASTER_IP * [BIGTOP-2122] - Add zeppelin packages * [BIGTOP-2123] - Add zeppelin Puppet recipes * [BIGTOP-2149] - Zeppeling 0.5.5 has been officially released. Change the source ref * [BIGTOP-2154] - spark-shell doesn't start anymore without Hive libs in the classpath * [BIGTOP-2166] - Zeppelin shouldn't be build against constant version of Ignite * [BIGTOP-2167] - Zeppelin interpreter list doesn't include Ignite * [BIGTOP-2169] - Zeppeling has more upstream dependencies * [BIGTOP-2174] - Bump up ignite-hadoop version to the latest 1.5.0-b1 * [BIGTOP-2185] - Exclude Zeppelin interpreter.json from RAT check * [BIGTOP-2219] - Comb the smoke-tests to make code base easier for refactoring * [BIGTOP-2226] - Remove gradle wrapper folder in smoke-tests * [BIGTOP-2234] - TestBlockRecovery incorrectly fails on a single node cluster * [BIGTOP-2235] - Allow smoke-tests to use filename regexps instead of explicite listings * [BIGTOP-2245] - TestFuseHCFS might fall on '+' op. involving String and GString; class name is wrong * [BIGTOP-2267] - Zeppeling 0.5.6 is out; let's bump it in our stack * [BIGTOP-2271] - Update maven rat config to match one from gradle * [BIGTOP-2277] - release assembly needs to include bigtop_toolchain * [BIGTOP-2278] - Wrap separate maven release steps into convenient gradle task ** Bug * [BIGTOP-1022] - Giraph build script should also specify ZooKeeper, HBase, and Hive versions * [BIGTOP-1318] - Consider not forking compute-classpath.sh for spark * [BIGTOP-1344] - spec files assume RPMs being built on Red Hat * [BIGTOP-1352] - Refactor puppet code for installing JDK7 * [BIGTOP-1658] - puppet recipe updates for latest spark (1.3+ ) * [BIGTOP-1690] - Puppet should automatically create data directories * [BIGTOP-1789] - Spark 1.3.0 incompatible with Hive 1.1.0 * [BIGTOP-1805] - Upgrade Hadoop to 2.7 if released * [BIGTOP-1838] - Pig build does not publish artifacts of build * [BIGTOP-1877] - Upgrade Crunch to 0.12.0 * [BIGTOP-1886] - Kafka server can not create a log-cleaner.log file * [BIGTOP-1892] - Current required version of gradle 2.4 is not used everywhere * [BIGTOP-1893] - Compilation of hadoop-yarn-client failed * [BIGTOP-1894] - Snappy development packages are missing from bigtop_toolchain * [BIGTOP-1896] - bigtop_toolchain broken bei ant update * [BIGTOP-1902] - typo in bigtop-deploy/vm/vagrant-puppet-vm/vagrantconfig.yaml * [BIGTOP-1905] - Update Hue build for the upcoming 3.9 release * [BIGTOP-1909] - Include compiled .mo files for HUE i18n * [BIGTOP-1913] - Update hive to 1.2.1 * [BIGTOP-1916] - Update Website for 1.0 * [BIGTOP-1917] - Simplify gradle creating apt/yum repositories for better CI * [BIGTOP-1936] - Provide JDK8 for Bigtop * [BIGTOP-1937] - redhat-lsb is required by kafka daemon * [BIGTOP-1938] - kafka packages /usr/bin on RPM * [BIGTOP-1940] - Consider removing tests expecting 'sudo' from skip-list * [BIGTOP-1946] - Missing ASL header in some of iTest files * [BIGTOP-1948] - Need to upgrade groovy-eclipse-batch as it keeps pulling from non-existing repo * [BIGTOP-1949] - Sqoop 1.4.5 artifacts aren't getting resolved in the release... * [BIGTOP-1950] - Upgrade maven-assembly plugin: StackOverFlowException is thrown * [BIGTOP-1951] - Fix licenses in the source files * [BIGTOP-1954] - Change the component name in the MAINTAINERS.txt * [BIGTOP-1956] - Multi RS HBase requires unique hbase.tmp.dir to be set for each RS on a node * [BIGTOP-1958] - Upgrade default repositories and docker images to 1.0 * [BIGTOP-1960] - The smoke-test wrapper in bigtop-deploy can only be used in redhat series of Linux * [BIGTOP-1963] - Upgrade Mahout to 0.11.0 * [BIGTOP-1966] - site's index.xml is malformatted * [BIGTOP-1967] - Update the front-page of the website with new CI hostname * [BIGTOP-1987] - Recover resources/kmeans_data.txt for Spark smokes * [BIGTOP-1996] - Dockerfiles for bigtop-slaves * [BIGTOP-1999] - website link to the release bits is broken and points to the top-level mirror's directory * [BIGTOP-2003] - Bigtop puppet fails to deploy on Ubuntu due to a hiera 1.3.0 bug * [BIGTOP-2004] - Download task fails with Gradle 2.6 * [BIGTOP-2007] - bigtop.mk version of the stack needs to be bumped to 1.1.0-SNAPSHOT * [BIGTOP-2008] - build.gradle has out-of-date version * [BIGTOP-2014] - [VM provisioner] Missing FQDN on Ubuntu causes puppet deployment malfunction * [BIGTOP-2016] - tez does not build on opensuse * [BIGTOP-2017] - Rebase bigtop-slaves on bigtop-puppet * [BIGTOP-2019] - BigPetStore Spark isn't compiling due to changes in SQL API * [BIGTOP-2026] - Phoenix build defines HBASE_VERSION in two different places. * [BIGTOP-2027] - Bump gradle version, the wrapper to 2.7 * [BIGTOP-2028] - Enhance puppet config of zookeeper to support kerberized clients * [BIGTOP-2032] - Tez install does not set up tez jars on hdfs causing Pig to fail * [BIGTOP-2037] - BIGTOP-1746 Added Files Without Apache License Headers * [BIGTOP-2038] - Pig destination name incorrect * [BIGTOP-2039] - Solr download URL is incorrect * [BIGTOP-2040] - Mahout can not be build with Maven 3.0.5 - build containers need to be upgraded * [BIGTOP-2041] - Spark pkg name is incorrect * [BIGTOP-2042] - Tachyon name is incorrect * [BIGTOP-2043] - Kafka source incorrect * [BIGTOP-2044] - Unnecessary printout has been introduced by BIGTOP-1494 * [BIGTOP-2046] - puppet module search path * [BIGTOP-2050] - Description of clean tasks have null entries * [BIGTOP-2052] - Remove obsolete environment variables * [BIGTOP-2053] - After rebasing on Hadoop 2.7.1 yarn module should be returned to ignite-hadoop build * [BIGTOP-2054] - Update Pig * [BIGTOP-2056] - Remove top-level check-env.sh * [BIGTOP-2057] - null check doesn't safeguard against non-existing values in the BOM config * [BIGTOP-2059] - Bump Ignite to 1.4 * [BIGTOP-2061] - toolchain is failing because add-apt-repository command isn't available off-hand * [BIGTOP-2062] - cluster.yaml declares undefined vars; apply is broken * [BIGTOP-2066] - init-hdfs.sh is broken by recent hadoop update * [BIGTOP-2068] - Cannot Build Bigtop-Utils packages * [BIGTOP-2071] - Gstring.empty doesn't exist * [BIGTOP-2074] - spark-worker doesn't start during deploy from master * [BIGTOP-2082] - Remove x86 Assembler Code from zookeeper * [BIGTOP-2083] - smoke-tests are still on hadoop 2.6.0 * [BIGTOP-2084] - rename all puppet modules to have an underscore rather a dash * [BIGTOP-2085] - gradle toolchain should install necessary puppet modules * [BIGTOP-2086] - Install essential puppet modules along with puppet itself * [BIGTOP-2088] - Support protobuf installation for OPENPOWER * [BIGTOP-2089] - Fix bigtop.sh generation * [BIGTOP-2090] - Remove left-over junk after BIGTOP-2053 * [BIGTOP-2094] - ignite-hadoop fails to deploy after BIGTOP-2084 * [BIGTOP-2098] - Update bigtop_deploy to current state of puppetize.sh * [BIGTOP-2114] - hive is broken after BIGTOP-2104 * [BIGTOP-2115] - phoenix is broken after BIGTOP-2104 * [BIGTOP-2120] - opensuse: Hue packages are broken after BIGTOP-1905 * [BIGTOP-2121] - Missing ':' in bigtop::roles description * [BIGTOP-2126] - Fix default repo locations in the deployment site.pp * [BIGTOP-2127] - opensuse: fix kerberos python module * [BIGTOP-2128] - ignite-hadoop man page still refers to the Incubator * [BIGTOP-2131] - [Docker] bigtop slaves images should be built without cache * [BIGTOP-2139] - crunch compile fails with Insufficient memory on POWER * [BIGTOP-2140] - hbase compile fails with Insufficient memory on POWER * [BIGTOP-2144] - Update default repos in Bigtop Provisioner * [BIGTOP-2147] - Minor nits in bigpetstore-spark/README.md * [BIGTOP-2148] - generator.SparkDriver outputs empty data in cluster mode * [BIGTOP-2150] - Reversing directional build behavior * [BIGTOP-2152] - Fix conflict with libressl-devel from mysql-devel * [BIGTOP-2155] - Fix Hue 3.9.0 build failed on Fedora * [BIGTOP-2156] - Fix Sqoop 1.4.5 build failed on OpenSuSE * [BIGTOP-2159] - unable to build RPM for zeppelin * [BIGTOP-2162] - phoenix-core-*-tests.jar should be located at $PHOENIX_HOME * [BIGTOP-2168] - A erroneous typo in FailureVars#loadProps method * [BIGTOP-2171] - shared gradle directory on slave containers should be writable for non-root users * [BIGTOP-2172] - get rid of gradlew inside of subprojects * [BIGTOP-2173] - smoke-tests need to pass-through system properties * [BIGTOP-2175] - BIGTOP_HOME should be either asserter or replaced with setup logic * [BIGTOP-2176] - Package version of ignite-hadoop has illegal "-" character * [BIGTOP-2177] - Build dependecies list has wrong component name for ignite * [BIGTOP-2181] - Setting Ignite version to use '.' broke Zeppelin build. * [BIGTOP-2187] - toolchain creates user with different ids on different systems * [BIGTOP-2188] - bigtop/puppet:ubuntu-14.04 image does not support ppc64le * [BIGTOP-2189] - bigtop/slave:ubuntu-14.04 image does not support ppc64le * [BIGTOP-2190] - libprotobuf8 2.5.0 installation fails on ubuntu 14.04 for ppc64le * [BIGTOP-2191] - openjdk-7 is missing in bigtop/slave:ubuntu-14.04-ppc64le docker image * [BIGTOP-2196] - Docker configuration for ubuntu-15.04-ppc64le * [BIGTOP-2198] - adding bigtop/slaves:ubuntu-15.04 support for ppc64le * [BIGTOP-2199] - Add change introduced by BIGTOP-2171 * [BIGTOP-2200] - Fix download path for protobuf ubuntu-15.04 on ppc64le * [BIGTOP-2201] - Fix BIGTOP-2200 again * [BIGTOP-2215] - Enable autopurge in zookeeper * [BIGTOP-2221] - rpmbuild is missing in bigtop/slaves:trunk-opensuse-13.2 docker image * [BIGTOP-2222] - Hadoop do-component-build should pull in and use BOM * [BIGTOP-2223] - .dockerignore doesn't have ASL header * [BIGTOP-2236] - GRADLE_USER_HOME should be set for bigtop/slaves images only * [BIGTOP-2237] - Nullify the standard output when generating gradle cache * [BIGTOP-2238] - Provisioner should propagate hdfs ssh keys for testing * [BIGTOP-2240] - add -XX:PermSize=1024m -XX:MaxPermSize=1024m to build environment * [BIGTOP-2242] - Running ./gradlew should only require JAVA_HOME for test task * [BIGTOP-2243] - :itest-common:test is broken without JAVA_HOME being set * [BIGTOP-2244] - CI provisioner tests are failing after BIGTOP-2227 change * [BIGTOP-2246] - Add smoke tests for HCFS * [BIGTOP-2247] - Expand HDFS smoke test suite * [BIGTOP-2252] - provisional hdfs ssh keys couldn't be found during deployment * [BIGTOP-2255] - bigtop/deploy:ubuntu-15.04 image does not support ppc64le * [BIGTOP-2256] - Promote return codes from docker to the CI while building images * [BIGTOP-2272] - moved hdfs ssh key are failing rat check * [BIGTOP-2275] - Update configuration files for jsvc * [BIGTOP-2276] - Zeppeling added war_tempdir location, which needs to be set * [BIGTOP-2281] - Add HIVE-12875 to Bigtop * [BIGTOP-2288] - Hadoop time-server fails to start * [BIGTOP-2299] - test resources are breaking rat ** Improvement * [BIGTOP-1126] - Add Hama to Bigtop * [BIGTOP-1131] - Update Build Requirements on our web pages * [BIGTOP-1309] - Gradle environment overhaul * [BIGTOP-1443] - Update front page of website * [BIGTOP-1809] - Remove gridgain-hadoop component once ignite-hadoop gets added * [BIGTOP-1888] - Upgrade Flume to 1.6.0 * [BIGTOP-1908] - Move bigtop-deploy's dockerfiles into a centralized docker place * [BIGTOP-1910] - Adjust the dependency for hue-beeswax * [BIGTOP-1914] - improve puppet README.md file * [BIGTOP-1915] - Upgrade Oozie to 4.2.0 * [BIGTOP-1920] - Include YCSB in Bigtop * [BIGTOP-1921] - Puppet recipe for YCSB * [BIGTOP-1923] - Bump Ignite to 1.2 with major fixes * [BIGTOP-1941] - Upgrade Phoenix to 4.4.0 * [BIGTOP-1942] - Upgrade Phoenix to 4.6.0 * [BIGTOP-1943] - Upgrade SCALA version to 2.10.4 * [BIGTOP-1944] - Upgrade Spark version to 1.5.1 * [BIGTOP-1955] - Upgrade Ignite Hadoop component version from 1.2.0 to 1.3.0 * [BIGTOP-1964] - Upgrade Tez version to 0.6.2 * [BIGTOP-1970] - Ignite IGFS now fully supports mutiltenancy: deployment should configure it with HDFS backing * [BIGTOP-1971] - Support Spark SQL CLI with Apache Hive out of the box * [BIGTOP-1974] - Revise SPARK_HOME/conf/spark-env.sh * [BIGTOP-1993] - Bump groovy to 2.4.4 in the development toolchain * [BIGTOP-1997] - Bump bigtop-groovy runtime to 2.4.4 * [BIGTOP-1998] - Toolchain installer needs to switch from CGI to Lua mirror selection * [BIGTOP-2005] - Remove SCALA_HOME requirement * [BIGTOP-2018] - Create a puppetizing script * [BIGTOP-2020] - Add Gradle RAT plugin to the top-level project * [BIGTOP-2063] - Provide default config to deploy hive on top of Ignite * [BIGTOP-2065] - Update deployment README.md to reflect on better deployment experience * [BIGTOP-2080] - Investigate removing Scala from the toolchain * [BIGTOP-2081] - implement a nexus docker container for CI * [BIGTOP-2091] - Build ignite-hadoop assembly with specific version of Spark * [BIGTOP-2102] - Upgrade YCSB to 0.4.0 * [BIGTOP-2103] - [Docker] Move bigtop/slaves image build to gradle * [BIGTOP-2110] - [Docker] Cache packages required by gradle to execute into bigtop/slaves images * [BIGTOP-2119] - Bump Mahout version to 0.11.1 * [BIGTOP-2141] - Have a way to specify alternative BOM file in the build time * [BIGTOP-2142] - Source cleanup: bigtop-repos shouldn't be top-level. * [BIGTOP-2143] - [Puppet] Automatically generate default repo * [BIGTOP-2153] - Simplify and complete storage dir creation * [BIGTOP-2163] - Add a phoenix-pherf sub-package for Phoenix * [BIGTOP-2164] - Phoenix Queryserver should write it's logs into /var/log/phoenix/ * [BIGTOP-2192] - Start generating ubuntu-14.04 deployment image * [BIGTOP-2194] - Add Ubuntu configuration to the docker provisioner * [BIGTOP-2239] - Smoke tests should have a single location for logger configuration * [BIGTOP-2259] - Check for smoke-test projects should be generalized ** New Feature * [BIGTOP-1149] - Package Kite * [BIGTOP-1746] - Introduce the concept of roles in bigtop cluster deployment * [BIGTOP-1769] - Zeppelin Integration * [BIGTOP-1976] - Replace Pair type with Apache Commons Lang Pair type ** Task * [BIGTOP-1701] - Upgrade to Hive 1.1.0 * [BIGTOP-1795] - Upgrade bigtop_toolchain to Maven 3.2.5 * [BIGTOP-1821] - Add smoke tests for Ignite Hadoop Bigtop module * [BIGTOP-1897] - Umbrella JIRA for CI overhaul * [BIGTOP-1925] - Add ywkim as a maintainer for YCSB * [BIGTOP-1953] - Upgrade bigtop_toolchain to Maven 3.3.3 * [BIGTOP-2069] - Update README.md to reflect CTR trial * [BIGTOP-2078] - Define Bigtop 1.1 release BOM * [BIGTOP-2146] - Upgrade bigtop toolchain ant version to 1.9.6 * [BIGTOP-2214] - Bump up ignite-hadoop version to the latest stable 1.5.0.final * [BIGTOP-2297] - Update provisioner pointers to (future) 1.1 release repos ** Test * [BIGTOP-2158] - Update README.md to reflect the acceptance of CTR model * [BIGTOP-2232] - Add smoke tests for HDFS * [BIGTOP-2249] - Add YARN smoke tests * Release 1.0.0 (2015-05-29) Release Notes - Bigtop - Version 1.0.0 ** Sub-task * [BIGTOP-1105] - Fix lintian errors in the hadoop package * [BIGTOP-1334] - Add DFS tests to TestCLI * [BIGTOP-1392] - Hive basic smoke test fails * [BIGTOP-1461] - Add a simple Hive validation to smoke-tests. * [BIGTOP-1465] - Include Sqoop1 in bigtop 0.9 * [BIGTOP-1468] - Include zookeeper 3.4.6 * [BIGTOP-1486] - Upgrade Hue to 3.7 * [BIGTOP-1535] - Add Spark ETL script to BigPetStore * [BIGTOP-1537] - [BigPetStore] Add Spark Product Recommender example * [BIGTOP-1543] - hive-0.14 in bigtop * [BIGTOP-1550] - Spark update to 1.1.0 * [BIGTOP-1563] - Puppet deployment needs to setup user hdfs keys for password-less logins * [BIGTOP-1571] - Remove Whirr from the BOM * [BIGTOP-1594] - Upgrade Pig to 0.14.0 * [BIGTOP-1609] - Use openjdk7 for CentOS * [BIGTOP-1615] - Tests dependencies are on old Hadoop versions (2.3.0, 2.4.1) * [BIGTOP-1632] - Support JMX monitoring when multiple HBase RS is brought up in a node * [BIGTOP-1649] - Upgrade Apache Flume to 1.5.2 * [BIGTOP-1707] - Upgrade Hadoop to 2.6.0 * [BIGTOP-1727] - Fix a build failure for Spark 1.2.1 * [BIGTOP-1728] - jackson dependency management for Spark 1.2.1 and Hadoop 2.6.0 * [BIGTOP-1738] - Update HBase because javadoc broken * [BIGTOP-1765] - Bump the stack version to 1.0 * [BIGTOP-1768] - Use openjdk in ubuntu14, Remove references to jdk6 * [BIGTOP-1842] - Remove gridgain-hadoop from BOM * [BIGTOP-1899] - Migrate CI master to EC2 instance donated by Amazon EMR team * [BIGTOP-1900] - Upgrade bigtop/slaves images on bigtop's dockerhub * [BIGTOP-1901] - Provide bigtop/deploy images on dockerhub for CI and users to consume docker provisioner * [BIGTOP-1947] - Fix RAT plugin configuration to be able to RAT-validate all published artifacts ** Bug * [BIGTOP-894] - Pig compilation fails on RPM systems on Bigtop trunk * [BIGTOP-965] - IntegrationTestHiveSmokeBulk not being run * [BIGTOP-972] - fix packagetest package remove logic * [BIGTOP-977] - hive smoke tests are full of internal infrastructure details irrelevant to Bigtop * [BIGTOP-1050] - Permissions on YARN LCE should be 4754 * [BIGTOP-1104] - Fix lintian errors * [BIGTOP-1121] - implement tests for HDFS snapshots * [BIGTOP-1135] - Hue has google analytics enabled by default * [BIGTOP-1194] - redhat-lsb-core is sufficient in spec files * [BIGTOP-1277] - Remove Windows files from unix packaging * [BIGTOP-1287] - Mahout smokes : Remove dirchlet/meanshift clustering * [BIGTOP-1327] - bigpetstore.arch is out of date * [BIGTOP-1356] - Generate hive.install on the fly * [BIGTOP-1357] - Get rid of HCatalog packaging code * [BIGTOP-1381] - Datafu and Spark .deb build is broken * [BIGTOP-1384] - Implement Gradle Wrapper for smoke tests and cleanup. * [BIGTOP-1395] - Simplifying non-component versions evaluation * [BIGTOP-1445] - Pig fails with clean .ivy and .m2 * [BIGTOP-1446] - Spark fails with clean .m2 * [BIGTOP-1451] - smoke-tests (gradle based) Mahout need to run a real smoke test. * [BIGTOP-1457] - Hue do-component-build does not set version properly * [BIGTOP-1458] - Maven build severly broken * [BIGTOP-1459] - Remove hadoop-client jar from packages that bundle it * [BIGTOP-1460] - maven dependencies broken * [BIGTOP-1462] - Failed to build spark by gradle * [BIGTOP-1467] - version of hadoop-auth in oozie is wrong * [BIGTOP-1472] - Hadoop RPM build is broken after BIGTOP-1462 * [BIGTOP-1487] - Documentation for use case and deployment models for failure tests. * [BIGTOP-1488] - bigtop_toolchain hardcode the apache mirror sites for downloading maven * [BIGTOP-1490] - Adding GridGain to BigTop * [BIGTOP-1491] - Update docker-puppet README * [BIGTOP-1497] - Add tachyon to bigtop * [BIGTOP-1502] - Improve puppet deployment for new gridgain component * [BIGTOP-1504] - Bigtop docker provision does not work on mac * [BIGTOP-1506] - bigtop-utils won't detect JAVA_HOME in some cases * [BIGTOP-1508] - fix Puppet warnings under Puppet 3 * [BIGTOP-1509] - Update deployment README after BIGTOP-1047 * [BIGTOP-1511] - TestCLI: make TestCLI compatible with Hadoop 2.4 (HADOOP-8691) * [BIGTOP-1513] - FailureExecutor.groovy is in the wrong module, causing compiler errors. * [BIGTOP-1521] - Bigtop smoke-tests hierarchy and fast failure * [BIGTOP-1522] - nodemanager deployment needs to include hadoop-mapreduce package * [BIGTOP-1523] - Gradle install for test artifacts uses wrong sequence of the targets * [BIGTOP-1524] - FailureExecutor breaks smoke tests : Smoke tests should run from source * [BIGTOP-1526] - property file for FailureVars can not be placed to {{/}} * [BIGTOP-1534] - Update README.md to reflect new build system and other changes * [BIGTOP-1541] - Support Debian jessie in Bigtop_toolchain; cleanup of pp files * [BIGTOP-1542] - Debian Packages will not build * [BIGTOP-1544] - [BigPetStore] Use of java.io.File and check for directory existence will fail with non-POSIX DFSs (HDFS, S3, etc.) * [BIGTOP-1548] - hue uses snapshot dependency to removed snapshot * [BIGTOP-1549] - spark compile broken * [BIGTOP-1551] - build stops because gradle does not handle symlinks well * [BIGTOP-1553] - puppet installation fails when components value is a single item instead of a list * [BIGTOP-1554] - bigtop-deploy/puppet/config/site.csv was wrongfully committed. * [BIGTOP-1562] - Critical tachyon pupet improvements. * [BIGTOP-1564] - docker-puppet doesn't use vagrantconfg.yaml for configuration parameters * [BIGTOP-1566] - Puppet README file needs to reflect multiple modules directoty requirement * [BIGTOP-1567] - puppet cannot install components whatever given to it * [BIGTOP-1568] - bigtop01 needs gradle * [BIGTOP-1570] - docker-puppet fails to deploy due to stdlib can not be found * [BIGTOP-1574] - Phoenix packaging requires build and classpath changes after PHOENIX-1455 * [BIGTOP-1579] - Implement patching for Bigtop * [BIGTOP-1580] - Improve Bigtop Toolchain: Versioning of Packages * [BIGTOP-1585] - test artifacts tasks include extra tasks for nonexisting components * [BIGTOP-1586] - BigPetStore-Spark only works on the East Coast . * [BIGTOP-1587] - Fix typos in install-tomcat.sh * [BIGTOP-1588] - Current Bigtop Pig does not build because of API Change in Hive * [BIGTOP-1589] - Prioritization is broken in bigtop-detect-javahome * [BIGTOP-1592] - Integration test IntegrationTestClusterFailures is failing * [BIGTOP-1596] - bigtop build fails because it tries to write to forrest installation * [BIGTOP-1604] - Create a MAINTAINERS.txt File * [BIGTOP-1605] - Tachyon RPM does not depend on bigtop utils * [BIGTOP-1606] - Tachyon webapp directory not located where expected * [BIGTOP-1610] - Fix /etc/profile.d/bigtop.sh scripts * [BIGTOP-1611] - mahoutsmokes aren't compiling * [BIGTOP-1618] - iTest integration tests need only to be bound to verify goal * [BIGTOP-1620] - Hadoop deb Packages for architecture "all" containing binaries * [BIGTOP-1626] - Add bmahe as a maintainer for Apache Flume * [BIGTOP-1633] - Pig compile fails again * [BIGTOP-1635] - hue compile on vanilla debian * [BIGTOP-1636] - Missing patches and files for hue * [BIGTOP-1639] - Add Olaf Flebbe to dev list * [BIGTOP-1642] - Restructure and enhance toolchain * [BIGTOP-1652] - Fix BigPetStore-Spark Data format regression * [BIGTOP-1654] - change some hadoop puppet variables from local to node scope * [BIGTOP-1656] - add tez variables to hadoop-env.sh * [BIGTOP-1657] - bigtop-deploy puppets to support tez * [BIGTOP-1660] - Upgrade to Hive-1.0 * [BIGTOP-1662] - puppet: Fix hadoop configuration file incompleteness due to hiera conversion * [BIGTOP-1663] - TestHadoopSmoke is failing because of improper set-up * [BIGTOP-1665] - Update Wiki pages on how to develop and build Bigtop * [BIGTOP-1668] - puppet: Adjust kerberos module for current Debian * [BIGTOP-1669] - puppet: Hadoop: Separate dependencies on Kerberos keytabs * [BIGTOP-1677] - Tez packaging is still failing * [BIGTOP-1678] - Use openjdk7 for AmazonLinux * [BIGTOP-1687] - Puppet: fix wrong array representation for hadoop-zookeeper::server::ensemble * [BIGTOP-1694] - puppet: Make httpfs subscribe to core-site and hdfs-site * [BIGTOP-1699] - sqoop does not build on debian * [BIGTOP-1703] - Limt override of JAVA_OPTS in install_solr.sh to cause minimal side-effects * [BIGTOP-1706] - Make TestBlockRecovery runnable under all users * [BIGTOP-1710] - Spark 1.1.0 does not build for DEB * [BIGTOP-1711] - add packages to compile hue * [BIGTOP-1712] - Specification for Docker images. * [BIGTOP-1722] - upgrade tachyon to 0.6.0 * [BIGTOP-1723] - update Tachyon Maintainers * [BIGTOP-1724] - Update centos vagrant box * [BIGTOP-1729] - Install Hive has typo in HCATALOG option * [BIGTOP-1731] - Gradle mustRunAfter doesn't work is expected * [BIGTOP-1732] - remove has_ssh in Docker provisioner * [BIGTOP-1733] - fix local_yum implementation * [BIGTOP-1745] - Add Mark Grover as committer on the website * [BIGTOP-1748] - Remove assert on fs.trash.interval from TestCLI and TestDFSCLI * [BIGTOP-1751] - Puppet deployment ignores bigtop_repo_uri variable * [BIGTOP-1756] - Add HADOOP_MAPRED_HOME property to common * [BIGTOP-1757] - bigtop_toolchain has to differentiate between centos6 and centos7 * [BIGTOP-1758] - Remove redundant property checks from TestHadoopExamples * [BIGTOP-1761] - Delete testConf.xml from hadoop-smoke resources * [BIGTOP-1763] - Handle broken symlinks when copying jars * [BIGTOP-1764] - Fix copying mapreduce jars to HDFS for Oozie * [BIGTOP-1774] - Update mailing list page * [BIGTOP-1775] - Fix typos and mistakes on mailing list page * [BIGTOP-1776] - Resolve an warning on packaging Hadoop RPM * [BIGTOP-1778] - spark-1.3.0 broke hive * [BIGTOP-1779] - giraph compile broken * [BIGTOP-1780] - bigtop_toolchain: JAVA_HOME incorrect, remove JAVA6_HOME * [BIGTOP-1781] - tachyon fails to build * [BIGTOP-1790] - Top level menu says 'Power By' - needs to be fixed * [BIGTOP-1793] - bigtop_toolchain: remove JAVA_HOME/bin from PATH * [BIGTOP-1798] - init-hdfs.sh uses groovy but no dependency in package description given * [BIGTOP-1804] - Not again: srpm target fails on SUSE * [BIGTOP-1808] - hive 1.0.0 : kerberos does not work correctly * [BIGTOP-1810] - Spark thriftserver service does not indicate success * [BIGTOP-1812] - fix bps pig integration test * [BIGTOP-1813] - Create /user/hbase HDFS directory * [BIGTOP-1814] - Puppet deployment code needs to work with ignite-hadoop component * [BIGTOP-1815] - Tez build is broken in the container * [BIGTOP-1816] - Puppet deploy is broken on Ubuntu * [BIGTOP-1817] - bigtop-utils deb is missing init-hcfs.groovy file * [BIGTOP-1818] - ignite-hadoop-service debian is missed; causing Puppet deploy to fail * [BIGTOP-1824] - Tachyon master and worker daemon script does not work propertly * [BIGTOP-1827] - Tachyon package name conflict on Debian/Ubuntu * [BIGTOP-1828] - Puppet: Tachyon does not get formatted before tachyon daemons are started up * [BIGTOP-1830] - Move apache-forrest installation from bigtop_toolchain to pig compile * [BIGTOP-1832] - hdfs-site.xml: update shortcut reader and remove obsolete shortcut reader users * [BIGTOP-1833] - Bump Ignite to 1.1.0 * [BIGTOP-1835] - Update project RDF file * [BIGTOP-1839] - Building RPM for Zookeeper 3.4.6 is broken * [BIGTOP-1840] - datafu does not pick up pig dependency from local repository * [BIGTOP-1841] - toolchain can not set java correctly in fedora * [BIGTOP-1845] - toolchain can not drop bigtop.sh in fedora because of puppet syntax error * [BIGTOP-1848] - spark deb packages broken * [BIGTOP-1849] - toolchain can not set java home correctly in CentOS 7 * [BIGTOP-1850] - Update Hue to 3.8 * [BIGTOP-1853] - Phoenix build does not properly override Hadoop version property * [BIGTOP-1855] - Remove Obsolete Jobtracker settings from mapred-site.xml * [BIGTOP-1857] - Support yarn-built-in proxy * [BIGTOP-1859] - Unable to build Hue packages on Amazon Linux * [BIGTOP-1860] - ignite-hadoop release URL should point to apache dist server, not a mirror * [BIGTOP-1861] - giraph does not build * [BIGTOP-1865] - circular dependency between bigtop-groovy and bigtop-utils * [BIGTOP-1866] - openssl-devel is required by hue RPM package * [BIGTOP-1867] - Upgrade Gradle version to 2.4 * [BIGTOP-1868] - Address the custom 'clean' warning * [BIGTOP-1869] - Hue 3.8.1 refinements * [BIGTOP-1870] - Latest version of Puppet::Apt doesn't work for our deployment recipes * [BIGTOP-1872] - Improve HUE puppet recipies * [BIGTOP-1874] - HBase build failed due to Codehaus repository is out of service * [BIGTOP-1875] - Oozie build failed due to Codehaus repository is out of service * [BIGTOP-1876] - Update puppet recipes for Sqoop2 * [BIGTOP-1879] - vagrant-puppet-vm is broken * [BIGTOP-1886] - Kafka server can not create a log-cleaner.log file * [BIGTOP-1893] - Compilation of hadoop-yarn-client failed * [BIGTOP-1894] - Snappy development packages are missing from bigtop_toolchain * [BIGTOP-1896] - bigtop_toolchain broken bei ant update * [BIGTOP-1902] - typo in bigtop-deploy/vm/vagrant-puppet-vm/vagrantconfig.yaml * [BIGTOP-1916] - Update Website for 1.0 * [BIGTOP-1937] - redhat-lsb is required by kafka daemon * [BIGTOP-1938] - kafka packages /usr/bin on RPM * [BIGTOP-1946] - Missing ASL header in some of iTest files * [BIGTOP-1949] - Sqoop 1.4.5 artifacts aren't getting resolved in the release... * [BIGTOP-1950] - Upgrade maven-assembly plugin: StackOverFlowException is thrown * [BIGTOP-1951] - Fix licenses in the source files * [BIGTOP-1958] - Upgrade default repositories and docker images to 1.0 ** Improvement * [BIGTOP-1047] - Support Puppet 3.x * [BIGTOP-1204] - Add support for AmazonLinux * [BIGTOP-1235] - Speed-up init-hdfs.sh for complete HCFS compliant provisioning * [BIGTOP-1275] - BigPetStore: Add all 50 states * [BIGTOP-1301] - Groovy 2.3.0 is out! Let's bundle it into the stack * [BIGTOP-1325] - Update TestHDFSCLI to include tests from upstream * [BIGTOP-1366] - Updated, Richer Model for Generating Data for BigPetStore * [BIGTOP-1388] - Use cluster failure tests during other tests with command line parametrization * [BIGTOP-1414] - Add Apache Spark implementation to BigPetStore * [BIGTOP-1423] - Add Groovy installation to the bigtop_toolchain * [BIGTOP-1449] - Add RC Testing support to vagrant recipe * [BIGTOP-1450] - Eliminate broken hive test artifacts in favor of smoke-tests. * [BIGTOP-1478] - Start tomcat watchdog as part of solr init scripts * [BIGTOP-1481] - Log files are cluttered with messages from tar * [BIGTOP-1484] - Include python in toolchain * [BIGTOP-1489] - move the tomcat_watchdog code out of install_solr.sh into a separate file in bigtop-utils in order to facilitate reusability * [BIGTOP-1495] - Remove make build system * [BIGTOP-1498] - Add RC Testing to Docker based Bigtop provisioner * [BIGTOP-1501] - come up with a way to use Jenkins plugins/Groovy console to create jobs * [BIGTOP-1510] - Minor README update aboud bigtop-deploy * [BIGTOP-1517] - refactor vagrant provisioners to be configurable by yaml file * [BIGTOP-1527] - Allow to fetch package's source code from Git * [BIGTOP-1545] - [BigPetStore] Bump version of BPS data generator library to 0.2.1 * [BIGTOP-1547] - Confirm tachyon functioning in vagrant recipes. * [BIGTOP-1555] - remove mapred-app from docs or code snippet (post-BIGTOP-1522), cleanup vagrant-puppet to core components. * [BIGTOP-1557] - website misses ci and announce mailing lists info * [BIGTOP-1558] - Make gradle easier to navigate * [BIGTOP-1559] - Tests are failing in Apache CI build * [BIGTOP-1569] - provide alias targets for building packages native to the OS * [BIGTOP-1576] - Add Vagrant installation to bigtop_toolchain * [BIGTOP-1578] - Refactor vagrant-puppet provisioner to have its configurations all come from yaml * [BIGTOP-1581] - Allow multiple Flume agents to be executed as a service using Bigtop init.d script * [BIGTOP-1583] - update the default yum repo in puppet site.pp * [BIGTOP-1590] - Add a functionality to run integration tests for itest * [BIGTOP-1591] - JarContentTest is failing on JDK7 * [BIGTOP-1593] - Fix types in FailureVars * [BIGTOP-1600] - Improve front page content of website * [BIGTOP-1601] - cleanup whitespaces across test-artifacts * [BIGTOP-1603] - Rename docker-puppet and vagrant-puppet to better indicate its function and difference * [BIGTOP-1612] - Update README for smoke-tests * [BIGTOP-1613] - Add .gitignore for vagrant-puppet * [BIGTOP-1614] - Add .gitignore file for docker-puppet * [BIGTOP-1621] - Migrate itest to gradle * [BIGTOP-1627] - Move enable_local_yum implementation to bigtop-deploy/utils/setup-env.sh to share with Docker provisioner * [BIGTOP-1634] - Puppet class parameter and hiera conversion * [BIGTOP-1640] - Add missing files for Apache Spark * [BIGTOP-1645] - Add subpackages for spark history server and thrift server * [BIGTOP-1651] - puppet: Format namenode and zkfc in noninteractive mode * [BIGTOP-1653] - Add queries for customer, state, and product statistics w/ d3 friendly JSON output to analytics phase. * [BIGTOP-1659] - gridgain-hadoop packages need to provide unversioned softlinks for accelerator jar files * [BIGTOP-1667] - Clean up and simplify the install phase for Apache Spark * [BIGTOP-1670] - puppet: Support Kerberos authentication on Hadoop component web GUIs * [BIGTOP-1674] - Account for BPS simulation "burn-in" time * [BIGTOP-1675] - Puppet: remove wheel group setting for bigtop_real_users * [BIGTOP-1676] - Puppet: add deb repo auto deploy * [BIGTOP-1681] - [BigPetStore] Clean up BPS SparkSQL analytics module, add new queries * [BIGTOP-1683] - puppet hadoop-env.sh: Add option to configure tez environment * [BIGTOP-1684] - puppet hadoop/yarn: Make container executor configurable * [BIGTOP-1685] - puppet hadoop: Remove useless use of explicit hiera lookup * [BIGTOP-1686] - puppet hadoop/mapred: Update and clean mapred template and namespace * [BIGTOP-1691] - Heira site.yaml template needs to have repo uri * [BIGTOP-1692] - hieradata/site.yaml is missing gridgain-hadoop component * [BIGTOP-1693] - Puppet stdlib should be automatically installed by toolchain * [BIGTOP-1696] - Provide build command to show BOM list * [BIGTOP-1697] - Bootstrap: hook up toolchain into gradle build * [BIGTOP-1698] - Following BIGTOP-1697 improve README entry on toolchain setup * [BIGTOP-1700] - Package YARN Timeline Server service for Apache Hadoop * [BIGTOP-1705] - puppet: Allow configuring which hue apps to install * [BIGTOP-1709] - Enable "hadoop26" profile for Apache Tez * [BIGTOP-1718] - add facility for printing out BOM in json format * [BIGTOP-1721] - Remove unnecessary files from Sqoop package * [BIGTOP-1725] - Exclude null values from BOM json output added in BIGTOP-1718 * [BIGTOP-1741] - Upgrade to Apache Phoenix 4.3.1 * [BIGTOP-1742] - Add ywkim as a maintainer for Apache Hive, Spark, Sqoop* and Phoenix * [BIGTOP-1743] - Add RJ Nowling as committer to website * [BIGTOP-1770] - Let the user specify which JDK package to be installed in vagrantconfig.yaml * [BIGTOP-1771] - Let the user specify what components to be tested when running smoke-tests by Docker/VM provisioner * [BIGTOP-1783] - Import BigPetStore Data Generator into BigTop * [BIGTOP-1787] - puppet: Update hue.ini to match Hue 3.7.1 template * [BIGTOP-1791] - Set group of /user/hue to "hue" in HDFS * [BIGTOP-1794] - Update README for bigtop_toolchain * [BIGTOP-1801] - Generalize subst pattern matching for BOM file * [BIGTOP-1803] - Overide Vagrant configurations from environment variables * [BIGTOP-1809] - Remove gridgain-hadoop component once ignite-hadoop gets added * [BIGTOP-1829] - Tachyon configuration files should reside in /etc/tachyon/conf instead of /etc/tachyon * [BIGTOP-1836] - Better UX for Docker provisioner by integrating it into gradle * [BIGTOP-1851] - Upgrade to HBase 0.98.12 * [BIGTOP-1862] - Add release download link to the front page of the website * [BIGTOP-1919] - Remove fixConditional methods in BPS data generator * [BIGTOP-1923] - Bump Ignite to 1.2 with major fixes * [BIGTOP-1932] - [BigPetStore] Add larger product collection * [BIGTOP-1939] - Enable basic hdfs-nfs gateway configuration ** New Feature * [BIGTOP-989] - Add Apache Kafka to Apache Bigtop * [BIGTOP-1417] - Dockerize the puppetized vagrant deployer * [BIGTOP-1448] - Include zookeeper C library in bigtop * [BIGTOP-1480] - Define BOM for 1.0 release of Bigtop * [BIGTOP-1552] - Add support for AmazonLinux to bigtop-deploy/puppet * [BIGTOP-1716] - Define SCALA_VERSION in bigtop.bom * [BIGTOP-1744] - Add BigPigPetstore File and REST Load Generation. * [BIGTOP-1806] - Integrate Apache Ignite (incubating) with BigTop * [BIGTOP-1918] - Add product enumeration to BPS data generator * [BIGTOP-1931] - Add multinomial product purchasing model to BPS Data Generator ** Task * [BIGTOP-1393] - Retire the make build system * [BIGTOP-1525] - FailureExecutor : Is it going to be maintained/used? * [BIGTOP-1556] - Cleanup some unused minor stuff. * [BIGTOP-1595] - kill list * [BIGTOP-1597] - Design flyers for upcoming SCALE conference * [BIGTOP-1638] - Add Dasha Boudnik to dev list * [BIGTOP-1648] - Update to Spark 1x to Spark 1.2.1 * [BIGTOP-1655] - Update VM and Docker provisioner to work with hiera converted puppet recipes * [BIGTOP-1726] - Confirm that Spark 1.2.1 runs properly in BigTop, and run the BPS integration test. * [BIGTOP-1755] - Upgrade to Spark 1.3.0 * [BIGTOP-1777] - Add cool ASCII-art logo of Bigtop to the build.gradle * [BIGTOP-1822] - Puppet apt module should be automatically installed by toolchain * [BIGTOP-1831] - Upgrade Mahout to 0.10 * [BIGTOP-1834] - Upgrade to Spark 1.3.1 * [BIGTOP-1854] - Trim Tachyon maintainers' list * [BIGTOP-1873] - Puppet apt module version must be 2.0.1 or above after BIGTOP-1870 * [BIGTOP-1897] - Umbrella JIRA for CI overhaul ** Test * [BIGTOP-1377] - Add a test case for performing block corruption recovery * [BIGTOP-1560] - Add a test case for performing block corruption recovery * [BIGTOP-1617] - Make clean-up commands in TestCLI more specific * [BIGTOP-1629] - Move testDFSCLI from hcfs to hdfs directory * [BIGTOP-1664] - make test unpack resources optional ** Wish * [BIGTOP-1619] - change hadoop namenode port to 8020 (apache default port) in puppet script * Release 0.8.0 (2014-09-06) Release Notes - Bigtop - Version 0.8.0 - SUB-TASK - [BIGTOP-1111] - Bump version of Hadoop to 2.2.0 - [BIGTOP-1165] - Ivy config for hive metastore looks incorrect, which breaks build on JDK 7 - [BIGTOP-1166] - Hive-jdbc doesn't compile on JDK 7, which breaks build - [BIGTOP-1183] - Rename spark package to spark-core - [BIGTOP-1184] - bump hadoop version to 2.3.0 - [BIGTOP-1202] - One-button-click way for installing all needed bits of testing framework - [BIGTOP-1203] - Update toolchaing to automatically install Gradle into dev env. - [BIGTOP-1208] - Add DFSIO test into longevity suite - [BIGTOP-1209] - Add SLive test into longevity suite - [BIGTOP-1242] - bump jsvc to 1.0.15 - [BIGTOP-1244] - bump flume to 1.5.0.1 - [BIGTOP-1278] - bump pig version to 0.12.1 - [BIGTOP-1279] - bump hive version to 0.12.0 - [BIGTOP-1280] - bump Oozie version to 4.0.1 - [BIGTOP-1281] - bump hbase version to 0.98.2 - [BIGTOP-1282] - bump Mahout version to 0.9 - [BIGTOP-1283] - bump Solr version to 4.6.0 - [BIGTOP-1284] - bump Spark version to 0.9.1 - [BIGTOP-1285] - Bump version of Phoenix to 4.0 - [BIGTOP-1291] - On a clean ~/.m2 hbase mvn install site will fail as install is executed first - [BIGTOP-1293] - Include Hive 0.13 into the stack - [BIGTOP-1302] - Update hbase smoke tests to escape Hbase's "event horizon" - [BIGTOP-1330] - bump Giraph version to 1.1.0 - [BIGTOP-1331] - bump Hue version to 3.6.0 - [BIGTOP-1340] - Hbase build has to use hadoop-two.version sys. prop - [BIGTOP-1350] - Bump version of Crunch to 0.10.0 - [BIGTOP-1354] - bump version of Solr to 4.9.0 - [BIGTOP-1426] - Bump version of Hadoop to 2.4.1 - [BIGTOP-1432] - Update HBase version to 0.98.5 in the BOM - [BIGTOP-1433] - Fix Hive build after BIGTOP-1429 - [BIGTOP-1434] - Maven build failing with NoClassDefFoundError at maven-site-plugin:3.0-beta-3:site (packaging-documentation) @ sqoop-docs - [BIGTOP-1435] - Fix Crunch build after BIGTOP-1429 - BUG - [BIGTOP-282] - the licensing status of the MovieLens data files needs to be cleared up - [BIGTOP-831] - Hadoop branch-2 (and trunk?) are failing when built with JDK7u11 - [BIGTOP-922] - Existing Puppet recipes are failing to initialize HDFS dictories on the first run - [BIGTOP-986] - install-flume.sh uses -x instead of -e - [BIGTOP-996] - Document steps needed to run BT tests on a fresh host - [BIGTOP-997] - TestCLI fails test id 5: comparator fails - [BIGTOP-1042] - Add support for Hadoop 2.1.x-beta - [BIGTOP-1048] - ZooKeeper installation use "zookeeper-server.pid" as default while ZooKeeper expects zookeeper_server.pid - [BIGTOP-1072] - Vagrant scripts for spinning up and "hydrating" bigtop vms - [BIGTOP-1092] - Add Hue documentation package - [BIGTOP-1097] - introduce bigtop-groovy package - [BIGTOP-1124] - solrctl should be able to auto-detect JAVA_HOME - [BIGTOP-1125] - Return value does not reflect status checks - [BIGTOP-1128] - FIX and modularize mahout sample data sets - [BIGTOP-1132] - Phoenix RPM specfile does not create symlinks - [BIGTOP-1141] - Add TestYarnNode to test yarn node - [BIGTOP-1143] - Oozie configuration file is incomplete - [BIGTOP-1146] - Flume NG package is broken because of org.hbase.asynchbase deps. mismanagement - [BIGTOP-1150] - update Spark debian packaging to the latest format - [BIGTOP-1153] - Crunch package doesn't depend on anything - [BIGTOP-1164] - Get whole Bigtop stack building on JDK 7 - [BIGTOP-1167] - got "Permission denied" when creating vagrant home folder in provision.sh - [BIGTOP-1168] - hue build fails (unresolvable maven dependencies) - [BIGTOP-1169] - Hue rpm build fails - [BIGTOP-1174] - missing dependency setting on init-hdfs.sh in puppet recipes - [BIGTOP-1181] - Add pyspark to spark package - [BIGTOP-1187] - Spark service packages have extraneous core in the name - [BIGTOP-1191] - master pom.xml file formatting is out of whack - [BIGTOP-1197] - iTest PackageManagerTests are failing - [BIGTOP-1199] - Makefile for deb packing needs to support zip archives - [BIGTOP-1205] - GRADLE_OPTS missing closing " in toolchain jenkins.sh.* - [BIGTOP-1217] - Spark test-artifact missing repository for install - [BIGTOP-1218] - Add JDK7 to Bigtop toolchain - [BIGTOP-1220] - Gradle task order isn't enforced properly with dependsOn - [BIGTOP-1221] - Expand and updated FUSE tests - [BIGTOP-1223] - TestSlive does not fully remove root directory after an iteration - [BIGTOP-1225] - TestHDFSQuota can fail if tests are run out of order - [BIGTOP-1226] - TestSLive setup() already being used, need to rename - [BIGTOP-1232] - test artifact build fails in io-longevity - [BIGTOP-1234] - TestFileAppend shouldn't reuse client during testMultipleOutputStreamFailure() - [BIGTOP-1236] - TestCLI expecting permissions incorrectly - [BIGTOP-1238] - TestCLI has misspelled 'NAMNEODE' that is used without protocol name - [BIGTOP-1239] - TestCLI: make ports optional in regex comparators - [BIGTOP-1246] - Fix the shell-object bug in HttpFs smoke tests - [BIGTOP-1250] - TestTextSnappy fails - [BIGTOP-1252] - Update ant from 1.9.2 to 1.9.3 - [BIGTOP-1253] - TestSLive calling setpDir() instead of setupDir() - [BIGTOP-1254] - add gradle specific files to .gitignore - [BIGTOP-1255] - init-hcfs.json needs to encode privileges as strings, not numbers. - [BIGTOP-1256] - Spark package deployment should reflect BIGTOP-1183 - [BIGTOP-1258] - Spark puppet deploy might fail as /etc/spark can not be created - [BIGTOP-1259] - spark-worker service should be started after spark-master if ran on the same node - [BIGTOP-1260] - Puppet deployment should reflect on the new name of spark package - [BIGTOP-1268] - Jenkins Job Bigtop-trunk-Hadoop failed from 3/15/2014 due to protoc version - [BIGTOP-1274] - yarn ResourceManager should be started before starting NodeManagers - [BIGTOP-1290] - Zookeeper build is failing on Ant 1.9.3 - [BIGTOP-1294] - asciidoc is missed from centos and sles environments of bigtop_toolchain - [BIGTOP-1295] - package.mk misteriously missing mkdir for RPMS that leads to SUSE build to fail - [BIGTOP-1300] - need to bump ant version to 1.9.4 in bigtop_toolchain - [BIGTOP-1303] - Pig 0.12.1 build is broken at the site target. Stack build can't proceed - [BIGTOP-1304] - bigtop_toolchain needs to install Scala 2.10 to support Spark build - [BIGTOP-1306] - RPM licensing metadata is inconsistent and wrong - [BIGTOP-1313] - Ant 1.9.4 update is missed from ant.pp recipe. - [BIGTOP-1319] - hive-rpm target is broken - [BIGTOP-1322] - hbase_install.sh shall be called with bash - [BIGTOP-1324] - Fix typo in testHDFSConf.xml of TestCLI - [BIGTOP-1336] - Puppet recipes failed to deploy kerberos enabled hadoop cluster - [BIGTOP-1337] - Logging success messages in service startup and shutdown only after verification. - [BIGTOP-1345] - remove bigtop-packages/src/root@mgrover-centos6-2.ent.cloudera.com - [BIGTOP-1346] - Generalize evaluation regexp in packages.gradle - [BIGTOP-1355] - Hive 0.13 deb build is broken - [BIGTOP-1358] - adjust toolchain to use JDK7 by default - [BIGTOP-1359] - fix a few issues with our puppet_toolchain code - [BIGTOP-1371] - Spark-master service fails to start due to missing /usr/lib/spark/work - [BIGTOP-1380] - Pig might run out of memory during site construction - [BIGTOP-1382] - Hadoop site.csv components additions: yarn & mapred-app - [BIGTOP-1390] - JDK Woes - [BIGTOP-1394] - package test manifest got out of date - [BIGTOP-1396] - update metadata for bigtop-groovy and phoenix - [BIGTOP-1397] - hive-hbase package is missing after move to the latest Hive - [BIGTOP-1398] - hadoop-hdfs-fuse can't be installed on Ubuntu 14.04 - [BIGTOP-1399] - add --help to alternative invocation to support newer Ubuntus - [BIGTOP-1400] - Whirr is failing with PermGen error: needs to be bumped up - [BIGTOP-1401] - Create man pages for YARN, HDFS and MAPRED. - [BIGTOP-1402] - Update bigtop gradle to 2.x - [BIGTOP-1405] - Gradle 2.0 installation from toolchain is broken - [BIGTOP-1415] - Remove redundant jars from spark package. - [BIGTOP-1418] - [BigPetStore] build.gradle - wrong variable name (hadoopClusterRuntime instead of runtime) - [BIGTOP-1420] - Update Phoenix version to 4.1.0 in the BOM - [BIGTOP-1421] - Phoenix build is missing some variables - [BIGTOP-1425] - gradle build can fail with NPE in an obscure way - [BIGTOP-1427] - HBase build should build hbase-***-hadoop2 by default - [BIGTOP-1428] - update tests to reflect new home for hbase user - [BIGTOP-1429] - Apache build of Bigtop test artifacts fails to find correct Hbase jars - [BIGTOP-1430] - incorrect version of spark-core prevents test artifacts from compiling - [BIGTOP-1438] - asciidoc is no longer needed by Sqoop - [BIGTOP-1439] - split jdk7 and jdk6 in Bigtop toolchain - [BIGTOP-1440] - a few usability updates to bigtop_toolchain - [BIGTOP-1441] - bigtop_toolchain fails on Ubuntu - [BIGTOP-1442] - need to add proper clean up for the toolchain puppet code - [BIGTOP-1444] - Phoenix RPM build is broken after BIGTOP-1421 - IMPROVEMENT - [BIGTOP-840] - Refactor Bigtop puppet code for better integration with newer Puppet and Whirr - [BIGTOP-848] - Allow to build stack on top of an arbitrary Hadoop SHA - [BIGTOP-952] - init-hdfs.sh is dog slow. Let's replace it with a direct HDFS API calls and better layout management - [BIGTOP-992] - Create HowToContribute wiki page - [BIGTOP-1019] - Remove mysql requirement constraint from sqoop tests - [BIGTOP-1026] - Big Top Mapred/Itest assertions should report specific commands which fail - [BIGTOP-1029] - Add support for HBase 0.96+ - [BIGTOP-1085] - spark packages needs to be split - [BIGTOP-1110] - Define BOM for 0.8.0 release of Bigtop - [BIGTOP-1130] - Update Build Requirements on our web pages - [BIGTOP-1137] - provide a way to request a major version of JDK - [BIGTOP-1138] - hadoop-conf-pseudo package uses the deprecated fs.default.name parameter - [BIGTOP-1139] - bigtop-tomcat and bigtop-jsvc don't depend on bigtop-utils - [BIGTOP-1144] - Need to update command 'puppet apply' in file bigtop-deploy/puppet/README.md - [BIGTOP-1147] - README.md improvments - [BIGTOP-1152] - speed up spark build - [BIGTOP-1155] - New logo ! - [BIGTOP-1161] - Have a way to dynamically select components to be deployed by Puppet recipes. - [BIGTOP-1163] - puppet apply shows db init error if the oozie server is already running - [BIGTOP-1171] - Puppetize the Vagrant deployer - [BIGTOP-1178] - Clusterize the puppetized vagrant deployer. - [BIGTOP-1195] - Higher level interface to smokes. - [BIGTOP-1200] - Implement Generic Text File to define HCFS filesystem semantics - [BIGTOP-1201] - Enhance (gradleize) the build to ease development, deployment; abstract implementation - [BIGTOP-1207] - Create a foundation layer to build longevity and performance tests for the stack - [BIGTOP-1210] - integrate BIGTOP-952 with plain text HCFS FS definition from BIGTOP-1200 - [BIGTOP-1213] - TestHadoopExamples smokes: orderering of tests; other improvements - [BIGTOP-1222] - Simplify and gradleize a subset of the bigtop smokes - [BIGTOP-1224] - Provide a simple order for tests - [BIGTOP-1241] - BIGTOP should update protobuf to 2.5 from 2.4.x - [BIGTOP-1257] - Make TestHadoopExamples modular enough to support longevity tests. - [BIGTOP-1269] - BigPetStore: Create build w/ gradle - [BIGTOP-1289] - Update component versions in the pom.xml (including hadoop) - [BIGTOP-1296] - Fix permission and Update README for vagrant-puppet recipe - [BIGTOP-1311] - allow easy site publishing - [BIGTOP-1312] - Add dynamic menus, update, links, modern CSS to bigtop site. - [BIGTOP-1314] - Deprecate make build - [BIGTOP-1316] - enhance Shell for better checking of return code & output logging - [BIGTOP-1321] - TestCLI: check assumptions before the test run - [BIGTOP-1342] - Make TestCLI usable for both HDFS and HCFS - [BIGTOP-1347] - Support better entropy performance on vagrant VMs - [BIGTOP-1348] - vagrant-puppet provisioning is too slow on installing packages - [BIGTOP-1372] - Bigtop needs feature that takes in multiple arguments to build specific components at command line - [BIGTOP-1374] - Add dynamic override of BOM variables into Gradle build - [BIGTOP-1391] - Improve README.md so that the markdown rendering is pretty and easy to read - NEW FEATURE - [BIGTOP-1039] - Add packaging for Shark analytics - [BIGTOP-1089] - BigPetStore: A polyglot big data processing blueprint - [BIGTOP-1188] - Smoke tests for HttpFs - [BIGTOP-1192] - Add utilities to facilitate cluster failure testing into bigtop-test-framework - [BIGTOP-1212] - Pick or build a framework for building fake data sets - [BIGTOP-1272] - BigPetStore: Productionize the Mahout recommender - [BIGTOP-1351] - Init script for HBase Thrift2 server is missing - TASK - [BIGTOP-236] - The user installed by hbase gets an incorrect home directory - [BIGTOP-1240] - Formatting Guidelines - TEST - [BIGTOP-1040] - Add integration tests for Shark component * Release 0.7.0 (2013-10-28) - SUB-TASK - [BIGTOP-932] - Allow puppet recipes to automatically pull in needed files from web - [BIGTOP-964] - Bump hive version to 0.11 - [BIGTOP-993] - Add packaging for Phoenix - [BIGTOP-1053] - Bump version of Hadoop to 2.0.6-alpha - [BIGTOP-1073] - Bump version of HBase to 0.94.11 - [BIGTOP-1074] - Bump version of Hue to 2.5.0 - [BIGTOP-1075] - bump version of Flume to 1.4.0 - [BIGTOP-1076] - bump version of Crunch to 0.7.0 - [BIGTOP-1077] - Bump version of Hive to 0.11.0 - [BIGTOP-1078] - bump version of Datafu to 1.0.0 - [BIGTOP-1079] - bump version of Solr to 4.5.0 - [BIGTOP-1083] - Spark version was changed with incubating process. - [BIGTOP-1101] - Bump version of HBase to 0.94.12 - BUG - [BIGTOP-990] - /usr/lib/bigtop-utils/ should be owned by bigtop-utils package - [BIGTOP-998] - Consolidate Hue packages - [BIGTOP-1005] - Create versionless symlinks in hadoop client directory - [BIGTOP-1006] - Explicitly state the working directory to be used in init scripts instead of relying on user's home directories - [BIGTOP-1008] - /usr/lib/bigtop-utils directory is not owned by bigtop-utils package in RPMs - [BIGTOP-1010] - libsnappy not included in the final hadoop RPM - [BIGTOP-1011] - bigtop-detect-javahome has a quirky search order - [BIGTOP-1017] - Revert a commit that introduces some unneeded dependencies - [BIGTOP-1027] - 'bigtop-detect-classpath' helper script missing from bigtop-utils - [BIGTOP-1033] - itest-common: fix debian install to avoid conffile prompt - [BIGTOP-1034] - Custom user's home directory's creation fails in init-hdfs.sh - [BIGTOP-1036] - Hue's oozie app package should depend on hue jobsub package - [BIGTOP-1037] - Provide a mechanism to control the sourcing of defaults files - [BIGTOP-1044] - hive and hive-jdbc versions should be kept in sync - [BIGTOP-1045] - Be consistent with shells, bigtop-detect-javahome, and bigtop-utils versions - [BIGTOP-1046] - Pig's use of BIGTOP_DEFAULTS_DIR is evaluated at the wrong time - [BIGTOP-1051] - Spark archive should come in tar.gz format - [BIGTOP-1052] - Increase environment configurability/debugging of Mahout Tests - [BIGTOP-1055] - Spark package doesn't create PREFIX/bin directory and sets wrong permissions - [BIGTOP-1059] - Add path option for pig.jar to pig smoke pom.xml - [BIGTOP-1070] - Add BIGTOP_CLASSPATH functionality to tomcat deployment - [BIGTOP-1071] - bigtop-detect-javahome should select the first possible candidate in the list - [BIGTOP-1081] - deb spark packing needs to set SOURCE_DIR - [BIGTOP-1082] - spark package tests are missing - [BIGTOP-1084] - spark build on CentOS 5 and SLES11 is broken - [BIGTOP-1086] - fix miscellaneous failures in package tests - [BIGTOP-1087] - flume rpm package needs to own /usr/lib/flume/tools - [BIGTOP-1088] - oozie packages need to own /etc/oozie/conf.dist/tomcat-deployment.{default,secure} - [BIGTOP-1090] - hue packaging needs to handle mutable bits (app.reg and hue.pth) - [BIGTOP-1094] - spark SPEC mistakenly re-defines alternatives command on SLES/OpenSUSE - [BIGTOP-1095] - oozie packaging doesn't remove tomcat alternatives - [BIGTOP-1096] - alternatives within the alternatives-managed sub-directory could be harmful - [BIGTOP-1098] - now that Spark has had its first incubator release we have to adjust bigtop accordingly - [BIGTOP-1099] - sync up the versions used for smoke tests with the BOM versions - [BIGTOP-1100] - the new Tomcat deployment style would benefit from a few usability tweaks - [BIGTOP-1103] - Remaining fixes of BIGTOP_DEFAULTS_DIR evaluation - [BIGTOP-1107] - Update to the official Hue 2.5.1 release - [BIGTOP-1108] - Solr 4.5.0 has been formally released and we should start using the official release bits - [BIGTOP-1112] - sqoop-client should require bigtop-utils - [BIGTOP-1114] - Hue metastore app is missing from core registration and Debian packages - [BIGTOP-1115] - update package test manifest for Bigtop 0.7.0 - [BIGTOP-1117] - hive-hcatalog integration has a few issues - [BIGTOP-1119] - phoenix DEB packages differ in a few ways from RPM counterparts - IMPROVEMENT - [BIGTOP-927] - Update puppet recipes to support Ubuntu - [BIGTOP-938] - Upgrade to Junit 4.11 - [BIGTOP-955] - HBase installation should advertise its location and configuration - [BIGTOP-1004] - Hadoop packages do not include version independent symlinks - [BIGTOP-1021] - Update HBase version to 0.94.9 - [BIGTOP-1031] - README has outdated/ambiguous information - [BIGTOP-1041] - Add Puppet support for Spark cluster deployment - [BIGTOP-1056] - some changes are introduced into Spark build: needs to be updated here - [BIGTOP-1063] - Add Scala 2.9.3 to bigtop-toolchain - [BIGTOP-1069] - Remove descriptions about JDK5 dependency - [BIGTOP-1080] - Change /usr/bin scripts to be alternatives instead of flat files - [BIGTOP-1106] - update our puppet deployment topology - NEW FEATURE - [BIGTOP-811] - Add /var/lib/bigtop as a location to install SQL connectors and other plug-ins - [BIGTOP-1023] - Define BOM for 0.7.0 release of Bigtop - TASK - [BIGTOP-715] - Add Spark packaging - [BIGTOP-939] - Make usage of bigtop-tomcat more dynamic - TEST - [BIGTOP-719] - Add TestTextSnappy to test hadoop fs -text with snappy compressed files - [BIGTOP-1030] - Develop integration tests for new Spark component - [BIGTOP-1057] - Add TeraGen / TeraSort Benchmakring * Release 0.6.0 (2013-06-10) - SUB-TASK - [BIGTOP-239] - HBase init scripts are shut down in the wrong order - [BIGTOP-812] - HttpFS is using Tomcat 6.0.32 which has numerous vulnerabilities - [BIGTOP-813] - add integration test for HCatlog - [BIGTOP-814] - add packaging test for HCatlog - [BIGTOP-822] - Bump version of Hadoop to 2.0.5 - [BIGTOP-824] - Bump version of Hive to 0.10 - [BIGTOP-832] - Bump version of Hue to 2.3.0 - [BIGTOP-833] - Update HCatalog packaging for Bigtop 0.6.0 integration - [BIGTOP-839] - add packaging test for Sqoop 2 - [BIGTOP-852] - Improve HDFS helper script - [BIGTOP-854] - add package testing manifest for HCatalog - [BIGTOP-855] - add integration tests for HCatalog - [BIGTOP-861] - init-hdfs.sh does not work from init scripts - [BIGTOP-862] - All the Apache Hadoop services ought to start nicely and in order - [BIGTOP-863] - verify the existence of the initscripts with typo - [BIGTOP-864] - update the appliance for the location of the init-hdfs.sh script - [BIGTOP-880] - Bump version of Datafu to 0.0.6 - [BIGTOP-891] - TestPigTest doesn't run against YARN cluster - [BIGTOP-915] - bump version of Oozie to 3.3.2 - [BIGTOP-916] - Bump version of Pig to 0.11.1 - [BIGTOP-917] - Bump version of Solr to 4.2.1 - [BIGTOP-926] - Bump version of Giraph to 1.0 - [BIGTOP-933] - Need to add ASF license text to bigtop-toolchain files - [BIGTOP-962] - Bump Sqoop version to 1.99.2 - [BIGTOP-963] - bump version of Whirr to 0.8.2 - [BIGTOP-980] - bigtop-toolchaing user module can't create 'jenkins' user - BUG FIXES - [BIGTOP-275] - scoping of variable in puppet code is incorrect - [BIGTOP-379] - Package testing for Bigtop 0.5.0 release - [BIGTOP-445] - yarn default file has the wrong name - [BIGTOP-463] - should we reconsider /usr/lib vs. /usr/libexec decision? - [BIGTOP-483] - Smoke test of Hadoop fails with clitest missing testConf.xml file - [BIGTOP-489] - TestCLI with two test cases fails ID - 146 and 147 related to refreshServiceAcl - [BIGTOP-504] - provide a functionality for detecting libjvm.so in bigtop-utils - [BIGTOP-545] - package testing manifest in trunk needs to be updated - [BIGTOP-553] - Metadata for packages needs to be harmonized between RPM and Debain - [BIGTOP-584] - Fix hive smoke test failures in trunk - [BIGTOP-624] - running start on a service already running should return success - [BIGTOP-637] - Update boxgrinder appliance for the coming release - [BIGTOP-786] - Not able to find wiki link on Bigtop homepage - [BIGTOP-804] - org.apache.bigtop.itest.pmanager.PackageManagerTest.testLookupGcc fails on some systems - [BIGTOP-808] - hadoop-conf-pseudo package breaks hadoop-conf alternatives during upgrades - [BIGTOP-809] - Remove harmless scary message from solr build - [BIGTOP-815] - Move the trunk forward now that 0.5.0 is released - [BIGTOP-818] - Updated list of supported platforms on Bigtop's homepage and readme file - [BIGTOP-823] - Remove obsolete slf4j removal in hive packaging - [BIGTOP-826] - Wrong return code returned when stopping Hue on RHEL/SUSE - [BIGTOP-827] - Ubuntu Install Instructutions Fail with a 404 - [BIGTOP-829] - Hue status returned wrong information on SLES - [BIGTOP-830] - Add $(APACHE_MIRROR) for bigtop-tomcat - [BIGTOP-834] - Remove workaround for MAPREDUCE-4814 in historyserver - [BIGTOP-835] - The shell exec method must have variants which have timeout and can run in background - [BIGTOP-836] - Hue status on SuSE doesn't work when run from outside of /usr/bin - [BIGTOP-837] - Oozie 3.3.0 can't be build against Hadoop 2.0.3 or later - [BIGTOP-841] - hadoop-conf-pseudo missed configuration for capacity scheduler and historyserver - [BIGTOP-842] - hadoop-mapreduce-historyserver service dumps some output to the console instead of its .out file - [BIGTOP-844] - hadoop rpm upgrade sequence is broken - [BIGTOP-846] - The helper script introduced in BIGTOP-547 doesn't create /user/$USER directories on HDFS - [BIGTOP-850] - Remove non-ASF repositories and infrastructure references from the project files - [BIGTOP-851] - Typo breaks detection of multiple libraries in bigtop-utils in some JVMs - [BIGTOP-853] - HBase test, TestLoadAndVerify does not work on secure clusters - [BIGTOP-857] - Inconsistent group ownership of /var/run/hadoop-hdfs between .deb and .rpm installs - [BIGTOP-858] - HBase tests are no longer compile with HBase 0.94 - [BIGTOP-859] - fix the Bigtop trunk package test - [BIGTOP-869] - hadoop preun should not stop services itself on behalf of its subpackages - [BIGTOP-870] - Bump version of Pig to 0.11 - [BIGTOP-873] - Hive needs Ivy cache cleared - [BIGTOP-874] - make test contracts show up in javadoc - [BIGTOP-875] - hadoop-smoke is broken because of missed dependency - [BIGTOP-876] - TestDFSAdmin in hadoop-smoke is outdated and still relies on presence of -upgradeProgress - [BIGTOP-877] - TestDFSAdmin in hadoop-smoke has to ensure its own environment, e.g. create HDFS paths, etc. - [BIGTOP-878] - Bump version of Oozie to 3.3.1 - [BIGTOP-879] - Bump version of Solr to 4.2.0 - [BIGTOP-883] - Add support for Oracle RPM Java 1.7 - [BIGTOP-884] - hive smoke tests can't be executed as a jar file isn't present anymore - [BIGTOP-885] - TestHiveSmokeBulk fails on Hive 0.9 - [BIGTOP-886] - Non-fatal errors when starting Hive daemons in EL5 - [BIGTOP-887] - Sqoop 2 should be consistent with Oozie / Bigtop for server plugins directory - [BIGTOP-888] - Oozie losing value of JAVA_HOME when spawning sub-shell - [BIGTOP-889] - pig smoke tests are no longer running - [BIGTOP-890] - Remove workaround for distributed cache problem from Oozie package - [BIGTOP-892] - hadoop packages misses capacity-scheduler configuration - [BIGTOP-894] - Pig compilation fails on RPM systems on Bigtop trunk - [BIGTOP-895] - A number of testcases in TestCLI are failing with (at least) Hadoop 2.0.3 and later - [BIGTOP-896] - Solr rpm requires redhat-lsb to run - [BIGTOP-899] - enabled deployment topology with 3 nodes - [BIGTOP-900] - update deployment configs for Kerberized installations - [BIGTOP-901] - Oozie smoke tests mightn't be able to obtain the jobtracker or namenode hostnames, using old config. names - [BIGTOP-902] - Hive uses same file for log4j and standard output - [BIGTOP-903] - crunch integration tests need to clean up output directory before each individual subtask is run - [BIGTOP-904] - mahout tests should default to /tmp as a temp location - [BIGTOP-905] - bump versions of test execution poms to match the BOM - [BIGTOP-906] - a few tests need artifacts compiled against Hadoop 2.X - [BIGTOP-907] - hadoop-mapreduce-historyserver should require hadoop-hdfs - [BIGTOP-910] - datanode init script cannot stop the process bcause it cannot find the pid file - [BIGTOP-912] - update HBase tests for 0.6.0 - [BIGTOP-914] - puppet needs to provision test users - [BIGTOP-918] - Move BOM version info to the root pom file - [BIGTOP-920] - Add HCatalog tests - [BIGTOP-923] - Packages that depend on bigtop-utils needs to specify >= 0.6 - [BIGTOP-928] - Hive can't be build against non-published Hadoop artifacts - [BIGTOP-929] - Pig component build needs to use internal resolvers - [BIGTOP-930] - init-hdfs.sh fails on execution - [BIGTOP-934] - Hidden cookie is being set by wiki.cloudera.org via http://blogs.apache.org/bigtop top page - [BIGTOP-935] - Use init script templates for hive - [BIGTOP-936] - Use init script templates for hbase - [BIGTOP-937] - Hive does not restart on Ubuntu - [BIGTOP-940] - fix issues with Hive package - [BIGTOP-945] - Service lock files need to match init script name - [BIGTOP-946] - solr-server runlevels are wrong in debian - [BIGTOP-947] - oozie init script does not use JAVA_HOME in /etc/default/bigtop-utils - [BIGTOP-948] - TestFileAppend and TestHDFSQuota tests need to be updated - [BIGTOP-949] - Add Sqoop tests - [BIGTOP-950] - race condition for output consumption in Shell code - [BIGTOP-951] - Hive Smoke's TestJdbcDriver fails - [BIGTOP-953] - Revert BIGTOP-835 and BIGTOP-950 - [BIGTOP-954] - metadada and user data for sqoop and hcatalog packages are wrong - [BIGTOP-956] - fix small issues with mahout, hue and oozie tests - [BIGTOP-957] - update deployment code for Sqoop2 - [BIGTOP-958] - update deployment code for HCatalog - [BIGTOP-959] - get rid of hadoop specific init.d.tmpl - [BIGTOP-960] - Remove workaround for HBASE-6263 in hbase-thrift start - [BIGTOP-968] - oozie is now part of the core hue - [BIGTOP-969] - a few files belong to more than one package - [BIGTOP-970] - update package test manifest and fix small issues in package metadata for Bigtop 0.6.0 - [BIGTOP-971] - take care of the rat licensing complaints - [BIGTOP-974] - Crunch download location is no longer /incubator - [BIGTOP-975] - HBase regionserver init script does not implement "condrestart" command - [BIGTOP-981] - Provide minimal documentation for puppet deploy config file - [BIGTOP-982] - hcatalog needs to be given HCAT_HOME and HIVE_HOME - [BIGTOP-983] - Oozie-client package ships its docs twice - [BIGTOP-984] - bigtop-utils doesn't detect JAVA_HOME for Oracle JDK 7 when package by Debian's tool - [BIGTOP-985] - Create Hive JDBC package - [BIGTOP-988] - HCatalog does not launch daemon process in a known directory - [BIGTOP-991] - crunch archive is incorrect - [BIGTOP-994] - hdfs-init missed a few permissions on the sub-directories - [BIGTOP-997] - TestCLI fails test id 5: comparator fails - [BIGTOP-1001] - need package tests for hive-jdbc package - [BIGTOP-1002] - Complete porting Sqoop import tests to Sqoop 2 - IMPROVEMENT - [BIGTOP-12] - Add HCatalog to Bigtop - [BIGTOP-547] - create an HDFS setup helper script - [BIGTOP-712] - Bigtop 0.5.0 release - [BIGTOP-713] - use newer debhelper and source format 3.0 (quilt) for Debian and Ubuntu packaging - [BIGTOP-756] - Provide the thrift API interface file in the HBase packages - [BIGTOP-763] - Add hdfs-journalnode to hadoop.spec - [BIGTOP-817] - Wiki breadcrumb still shows Bigtop(incubating) - [BIGTOP-821] - Puppet README's Usage section cuts off because of angle bracket delimiter - [BIGTOP-838] - Both conf.empty and conf.pseudo packages don't seem to include a sample hadoop-env.sh file - [BIGTOP-843] - It would be really cool if bigtop-detect-javahome were smarter and didn't pick a 32-bit JVM, or a 1.7 JVM set with default - [BIGTOP-882] - Upload content of Oozie sharelib to HDFS - [BIGTOP-893] - migrate puppet code to use init-hdfs.sh - [BIGTOP-931] - a few improvements to secure puppet deployment code - [BIGTOP-961] - hcatalog should be re-using hive user when running - [BIGTOP-967] - package removal is currently missing from our package tests - [BIGTOP-973] - Improve error diagnostics when sed'ing Hadoop version in Oozie package - [BIGTOP-995] - JDK bug 6979329 requires a redundant kinit -R - [BIGTOP-1000] - Remove remaining references to the Incubator - NEW FEATURE - [BIGTOP-732] - Support running multiple HBase region servers - [BIGTOP-881] - Add support for Oozie to be configured with SSL (HTTPS) - TASK - [BIGTOP-118] - Check to ensure our build system requirements are actually declared properly in the packaging. - [BIGTOP-681] - Package tests needed for new DataFu package - [BIGTOP-720] - Build environment automation - [BIGTOP-760] - Push artifacts from non-Maven projects into local Maven cache - [BIGTOP-805] - Add Sqoop 2 to bigtop - TEST - [BIGTOP-621] - Add test for distcp - intra cluster - [BIGTOP-625] - Add HDFS Append Test - [BIGTOP-693] - Add hadoop-fuse-dfs test - [BIGTOP-728] - add datafu integration test * Release 0.5.0 (2012-12-25) - SUB-TASK - [BIGTOP-39] - Include Apache Avro in Bigtop - [BIGTOP-41] - Add Apache Thrift to Bigtop - [BIGTOP-65] - Package Apache Forrest - [BIGTOP-225] - Exit consistently in init scripts for stop on stopped service - [BIGTOP-232] - create an example .ks file for Bigtop - [BIGTOP-237] - Oozie init scripts are way too verbose - [BIGTOP-239] - HBase init scripts are shut down in the wrong order - [BIGTOP-240] - hbase initscripts should be more explicit - [BIGTOP-302] - 1. Start a separate Wiki page on Installing Bigtop in VMs (VirtualBox) (make sure to record the fact that one needs >> 2GB of RAM) 2. Record the experience with 32bit OSes - [BIGTOP-380] - Initscripts have misleading messages - [BIGTOP-533] - BigTop Hama Integration Tests - [BIGTOP-551] - Updates for packaging manifest - [BIGTOP-612] - Add Crunch to Bigtop - [BIGTOP-692] - Bump Whirr to 0.8.1 - [BIGTOP-695] - Make variables used in flume build/install scripts consistent - [BIGTOP-696] - Make variables used in oozie build/install scripts consistent - [BIGTOP-697] - Make variables used in zookeeper build/install scripts consistent - [BIGTOP-700] - Make variables used in whirr build/install scripts consistent - [BIGTOP-702] - Make variables used in datafu build/install scripts consistent - [BIGTOP-703] - Make variables used in jsvc build/install scripts consistent - [BIGTOP-704] - Make variables used in pig build/install scripts consistent - [BIGTOP-705] - Make variables used in giraph build/install scripts consistent - [BIGTOP-706] - Make variables used in hive build/install scripts consistent - [BIGTOP-707] - Make variables used in tomcat build/install scripts consistent - [BIGTOP-711] - update hadoop version to 2.0.2-alpha - [BIGTOP-714] - HDFS Fuse build broken for 0.23.3 - [BIGTOP-733] - Bump Hue to 2.1.0 - [BIGTOP-735] - Upgrade to pull Solr 4.0 when it's officially released - [BIGTOP-736] - When Solr is integrated, we'll need some tests. - [BIGTOP-744] - add package test for Solr - [BIGTOP-750] - Bump Zookeeper to 3.4.5 - [BIGTOP-751] - Bump HBase to 0.94.2 - [BIGTOP-752] - Bump Sqoop to 1.4.2 - [BIGTOP-753] - Bump Flume to 1.3.0 - [BIGTOP-765] - Create deployment code for Solr - [BIGTOP-791] - Bump Oozie to 3.3.0 - BUG - [BIGTOP-51] - Layout of Bigtop repository can be optimized - [BIGTOP-79] - Audit RPM package dependencies for programs invoked from install/uninstall scripts - [BIGTOP-98] - Ability to force ivy/maven version inter-dependency needs to be implemented - [BIGTOP-105] - hadoop services can not be started with the default configs in place - [BIGTOP-106] - review and cleanup install/unistall/upgrade time dependencies - [BIGTOP-107] - make Bigtop packaging work in kick-start environment - [BIGTOP-135] - make jsvc use detection more robust on SLES - [BIGTOP-168] - all of the packages that link to hadoop-core need to also link commons-configuration-1.6.jar - [BIGTOP-172] - Hadoop init.d scripts are flaky on stop - [BIGTOP-177] - Remove or fix versioned dependencies between packages of Bigtop - [BIGTOP-187] - some packages ship log files - [BIGTOP-200] - /var/lib/hadoop is set to 777 on installation it should just be group writable - [BIGTOP-213] - We should use another group than hadoop for system groups - [BIGTOP-218] - we need to figure out which compressors/codecs need to be pulled into Bigtop - [BIGTOP-219] - Pig/HBase integration is not working - [BIGTOP-222] - some man pages in /usr/share/doc/flume - [BIGTOP-223] - install/ initscripts fixes - [BIGTOP-224] - Some configuration files have their permissions set to 755 - [BIGTOP-226] - everything in /usr/lib/pig is set to 755. Non executable files should just be 644 - [BIGTOP-227] - whirr has all its files (except its man page) set as 755 - [BIGTOP-228] - Whirr should have a symlink to the hadoop's jars it needs - [BIGTOP-229] - pig package install an alternative with an inconsitent name - [BIGTOP-230] - removing packages displays the following extraneous messages - [BIGTOP-256] - Lintian Errors/Warnings - Hadoop Package - [BIGTOP-275] - scoping of variable in puppet code is incorrect - [BIGTOP-279] - we shouldn't be using properties that have been deprecated in config/puppet - [BIGTOP-282] - the licensing status of the MovieLens data files needs to be cleared up - [BIGTOP-292] - BIgtop SRPMs can not be rebuilt - [BIGTOP-307] - README documentation is inaccurate - [BIGTOP-311] - hadoop does not build in off-line mode - [BIGTOP-326] - puppet modules need to be renamed to align with renaming that happens on the package front - [BIGTOP-330] - hadoop 0.23 pseudo conf needs to set more properties to avoid using /tmp as its datadir - [BIGTOP-335] - sqoop-metastore service fails to start because it ends up using /usr/lib/hadoop/bin/hadoop - [BIGTOP-340] - wrapper scripts would benefit from being explicit - [BIGTOP-347] - we need to get rid of workaround for HDFS-1943 in hadoop init.d scripts - [BIGTOP-358] - now that hadoop packages have been split we have to update the dependencies on the downstream packages - [BIGTOP-368] - /usr/bin/hadoop sources /etc/default/yarn unconditionally - [BIGTOP-377] - Clean up packages - [BIGTOP-379] - Package testing for Bigtop 0.5.0 release - [BIGTOP-385] - new artifacts in Hadoop 0.23 need to be properly packaged - [BIGTOP-395] - flume-ng init.d scripts seem to be unhappy on Debian - [BIGTOP-425] - HMaster dies from RemoteException: Server IPC version 5 cannot communicate with client version 4 - [BIGTOP-436] - flume-node stop seems to mistarget some other java process on lucid - [BIGTOP-445] - yarn default file has the wrong name - [BIGTOP-463] - should we reconsider /usr/lib vs. /usr/libexec decision? - [BIGTOP-471] - hive server setup after installation fails on creating /var/lock/subsys directory - [BIGTOP-483] - Smoke test of Hadoop fails with clitest missing testConf.xml file - [BIGTOP-486] - separate principals from keytabs in our kerberos module - [BIGTOP-489] - TestCLI with two test cases fails ID - 146 and 147 related to refreshServiceAcl - [BIGTOP-490] - SmokeTest-Mahout test failures - [BIGTOP-492] - make our launcher scripts recognize cascading defaults - [BIGTOP-504] - provide a functionality for detecting libjvm.so in bigtop-utils - [BIGTOP-511] - Add debian repo to bigtop puppet - [BIGTOP-520] - HBase Debian builds run do-component-build twice - [BIGTOP-525] - make sure yarn.dispatcher.exit-on-error is set to true - [BIGTOP-531] - package metadata check error: when description is empty, the summary will append text "description" at the end of summary - [BIGTOP-542] - Issue when installing the new zkfc - [BIGTOP-545] - package testing manifest in trunk needs to be updated - [BIGTOP-549] - update puppet code for the latest state of HA in Hadoop - [BIGTOP-553] - Metadata for packages needs to be harmonized between RPM and Debain - [BIGTOP-566] - Flume NG pkg init script should allow user to customize the location of the conf dir - [BIGTOP-581] - Need to move iTest website to a correct location - [BIGTOP-584] - Fix hive smoke test failures in trunk - [BIGTOP-591] - libhdfs0-dev is missing files - [BIGTOP-615] - Add support for Ambrose Pig job visualization - [BIGTOP-624] - running start on a service already running should return success - [BIGTOP-634] - Weird output when doing an init action on oozie when it is already running - [BIGTOP-637] - Update boxgrinder appliance for the coming release - [BIGTOP-644] - fix the runlevels for a few packages in yum/apt/zypper - [BIGTOP-648] - hbase-thrift cannot be started properly - [BIGTOP-654] - Set nproc for HDFS user - [BIGTOP-658] - Move hbase dependency in hive out of the main package - [BIGTOP-662] - cannot start hue-server - [BIGTOP-663] - cannot start sqoop-metastore in Ubuntu precise - [BIGTOP-664] - hive-metastore script has error when execute 'service hive-metastore status' in Ubuntu precise - [BIGTOP-667] - start hadoop-httpfs in SLES return 3 instead of 0 - [BIGTOP-669] - Add DataFu to Bigtop distribution - [BIGTOP-676] - Update HBase version to 0.94.1-rc0 - [BIGTOP-679] - fuse_dfs binary has moved due to HDFS-3537 - [BIGTOP-684] - copyright header in bigtop-packages/src/common/hadoop/conf.secure/core-site.xml is in wrong place - [BIGTOP-686] - Apache Flume 1.2.0 went out so we should upgrade to it - [BIGTOP-689] - Oozie use of /var/tmp in some CentOS versions create issues - [BIGTOP-691] - flume gets killed too fast when the service is asked to stop - [BIGTOP-694] - Some HBase Scripts Need To Be Excluded From Packaging - [BIGTOP-699] - hadoop-fuse-dfs fails to find libjvm.so when JAVA_HOME happens to be a symlink - [BIGTOP-701] - Bigtop Hadoop package does not include bash tab completion - [BIGTOP-708] - modify Hive integration test to depends on the correct jar file - [BIGTOP-721] - improve the package daemon status check, check twice by some delay if status doesn't match expected value - [BIGTOP-725] - package service test TestServices should only do config once for a group of services - [BIGTOP-729] - zookeeper initd script status doesn't work - [BIGTOP-734] - Proposed fixes to Hive logging configuration - [BIGTOP-740] - improve the package file content tests to ignore platform specific file names - [BIGTOP-745] - fix incorrect javadocs in test framework - [BIGTOP-754] - libhdfs0-dev package is empty - [BIGTOP-755] - YARN_HOME is now HADOOP_YARN_HOME - [BIGTOP-758] - [Pig] Override hadoop-core and hadoop-test artifact versions only if Hadoop < 2.x - [BIGTOP-759] - Flume do-component-build should set hbase.version according to BOM - [BIGTOP-762] - Updates for building Flume versions >= 1.2 - [BIGTOP-766] - Make it possible to override Apache URLs for the builds that require them - [BIGTOP-768] - Bump the version of Groovy to 1.8 - [BIGTOP-774] - solr build fails on Debian - [BIGTOP-775] - In Groovy 1.8 the behavior of a getters of static fields changed slightly - [BIGTOP-776] - A few updates to Oozie related packaging code to accomodate Oozie's TLP status - [BIGTOP-778] - move Solr app to under /solr in Tomcat deployment and provide more knobs - [BIGTOP-780] - cannot install hive-server in SLES - [BIGTOP-781] - Bigtop packages seem to have a bunch of parasitic provides - [BIGTOP-782] - 'service hue status' still show 'failed' after hue is started in SLES - [BIGTOP-783] - 'service hue stop' cannot stop the hue service in SLES - [BIGTOP-785] - clean up build-time dependencies in our packages - [BIGTOP-788] - Oozie RPM build error due to lack of usr/lib/oozie/{libserver,oozie-server,libext} - [BIGTOP-789] - remove shadow-utils dependency from zookeeper - [BIGTOP-793] - add crunch integration tests - [BIGTOP-794] - puppet code around hadoop_head_node breaks on newer puppets - [BIGTOP-795] - need to abstract away the notion of Bigtop puppet repo - [BIGTOP-796] - hive install tries to tickle HDFS and it shouldn't - [BIGTOP-797] - provide a way to 'rsync' files into HDFS during puppet deployment - [BIGTOP-798] - introduce a fatjar collection of all the Bigtop integration tests and all their dependencies - [BIGTOP-800] - BIGTOP_JSVC is lacking ARCHIVE designation - [BIGTOP-801] - prioritize jdk over jre in bigtop-utils javadetection - [BIGTOP-803] - somehow puppet gets confused by the hadoop fs output - IMPROVEMENT - [BIGTOP-12] - Add HCatalog to Bigtop - [BIGTOP-22] - Unify init scripts - [BIGTOP-58] - Add man page for Mahout - [BIGTOP-60] - Add normal rpmbuild options for make rpm - [BIGTOP-78] - Misc improvments - [BIGTOP-117] - Document incompatible API changes between releases - [BIGTOP-153] - Build should detect invalid archives in the dl directory - [BIGTOP-204] - Add information about supported projects and criteria for support to Apache Bigtop website - [BIGTOP-233] - Ubuntu integration: ufw application profiles - [BIGTOP-245] - Improve Bigtop layout - [BIGTOP-263] - we need a way to provide supervision for long running daemon applications - [BIGTOP-284] - It would be nice to integrate Nutch crawling as one of the MR (and perhaps HBase) tests for Bigtop - [BIGTOP-304] - document test creation and test execution steps in Bigtop - [BIGTOP-351] - Make HBASE_HEAPSIZE manageable via puppet - [BIGTOP-375] - Publish a table of component dependency versions - [BIGTOP-410] - Build secure version of HBase - [BIGTOP-433] - Make variables used in build/install scripts consistent - [BIGTOP-451] - ooziedb.sh script should run as the unix user that runs oozie server - [BIGTOP-452] - oozie installation should offer to run ooziedb.sh - [BIGTOP-456] - Consider splitting homedir between mapred and hdfs users? - [BIGTOP-476] - Improvement of BigTop iTest framework - [BIGTOP-497] - add bash tab completion to our hadoop packages - [BIGTOP-517] - Flume packaging should use the term "agent" instead of "node". - [BIGTOP-536] - provide suitable title in the