*************** Building Bigtop *************** Build Requirements: =================== * A stock image of a Linux Distribution, supported are - Debian 8 - Centos 6,7 - Ubuntu 14.04 LTS - SLES 11 SP3 - Fedora 20 - openSuSE 13.1 * A fast Internet Connection, since compile process will download a lot * Recommended is to install dependencies via puppet. Either - Read bigtop_toolchain/README.md or - install puppet and run puppet apply --modulepath= -e "include bigtop_toolchain::installer" Building ======== For Debian, Ubuntu Bigtop supports the deb packaging format and apt repositories For Centos, Fedora Bigtop supports rpm packaging and yum repositories For openSuSE, SLES Bigtop supports rpm packaging but zypper seems not supported. Bigtop consists of many packages which should be compiled in order: See bigtop.mk for the list of packages and names. For doing a full recompile do a gradle deb or gradle rpm depending on Linux distribution. If you want to create a repository gradle apt or gradle yum Build results are in output/ The $(HOME)/.m2 will contain several GB of downloads afterwards as collateral damage. Each package will have a "-1" release part by default. Create updated Packages ======================= In order to create releases where one can update machines (for instance in production) it is advisable to increase the release part for every rebuild. This can be done by using the BIGTOP_BUILD_STAMP to set the release part of the version of every package. For instance by default a package hadoop-2.4.1-1.deb will be created. If you need to fix this package and recompile it run: BIGTOP_BUILD_STAMP="2" gradle hadoop-apt This updates the package to hadoop-2.4.1-2.deb in repository, removing the old one. Please do not use a hyphen "-" in the BIGTOP_BUILD_STAMP breaking the algorithms apt and yum are using for calculating dependencies. Integration into a CI system: jenkins ===================================== A nightly build job can be created with gradle clean BIGTOP_BUILD_STAMP="nightly-${BUILD_NUMBER}" gradle apt|yum For more background information on BIGTOP_BUILD_STAMP see ticket BIGTOP-1580. Using a different JVM ===================== By default, bigtop is compiled with a JDK version 8. If you want to change it, for example to use JDK version 7, you just have to define the variable BIGTOP_JDK: BIGTOP_JDK=7; export BIGTOP_JDK ./gradlew deb Note on LC_ALL/LANG =================== Compilation may fail if your locale is not in english. You might for example see some strange errors with dpkg-buildpackage. In that case, you should reset the language variables before compiling: LC_ALL=C; export LC_ALL LANG=C; export LANG ./gradlew deb