1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
|
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under
the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may
obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to
in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under
the License. -->
<document xmlns="http://maven.apache.org/XDOC/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
<properties>
<title>Apache Bigtop</title>
</properties>
<body>
<section name="Apache Bigtop">
<p>
<b>
Bigtop is an Apache Foundation project for Infrastructure Engineers and Data Scientists looking for comprehensive packaging, testing, and configuration of the leading open source big data components.
</b>
Bigtop supports a wide range of components/projects, including, but not limited to, Hadoop, HBase and Spark.
</p>
<br />
<table>
<tr>
<th>Packaging</th>
<th>Smoke testing</th>
<th>Virtualization</th>
</tr>
<tr>
<td>
Bigtop packages Hadoop RPMs and DEBs, so that you can manage and maintain your Hadoop cluster.
</td>
<td>
Bigtop provides an integrated smoke testing framework, alongside a suite of over 50 test files.
</td>
<td>
Bigtop provides vagrant recipes, raw images, and (work-in-progress) docker recipes for deploying
Hadoop from zero.
</td>
</tr>
</table>
<br />
<p>
Bigtop components power the leading Hadoop distros and support many Operating Systems, including Debian/Ubuntu, CentOS, Fedora, SUSE and many others.
</p>
<p>
Bigtop includes tools and a framework for testing at various levels (packaging, platform, runtime, etc.) for both initial deployments as well as upgrade scenarios for the entire data platform, not just the individual components.
</p>
<br />
<br />
<subsection name="Tool requirements for building Bigtop"></subsection>
<table>
<tr>
<th>On all systems</th>
<th>Also on RPM-based systems</th>
<th>Also on DEB-based systems</th>
</tr>
<tr>
<td>
<ul>
<li>Java JDK 1.6</li>
<li>Apache Ant</li>
<li>Apache Maven</li>
<li>wget</li>
<li>tar</li>
<li>git</li>
<li>subversion</li>
<li>gcc</li>
<li>gcc-c++</li>
<li>make</li>
<li>fuse</li>
<li>protobuf-compiler</li>
<li>autoconf</li>
<li>automake</li>
<li>libtool</li>
<li>sharutils</li>
<li>xmlto</li>
</ul>
</td>
<td>
<ul>
<li>lzo-devel</li>
<li>zlib-devel</li>
<li>fuse-devel</li>
<li>openssl-devel</li>
<li>python-devel</li>
<li>libxml2-devel</li>
<li>libxslt-devel</li>
<li>cyrus-sasl-devel</li>
<li>sqlite-devel</li>
<li>mysql-devel</li>
<li>openldap-devel</li>
<li>rpm-build</li>
<li>createrepo</li>
<li>redhat-rpm-config (RedHat/CentOS only)</li>
</ul>
</td>
<td>
<ul>
<li>libxslt1-dev</li>
<li>libkrb5-dev</li>
<li>libldap2-dev</li>
<li>libmysqlclient-dev</li>
<li>libsasl2-dev</li>
<li>libsqlite3-dev</li>
<li>libxml2-dev</li>
<li>python-dev</li>
<li>python-setuptools</li>
<li>liblzo2-dev</li>
<li>libzip-dev</li>
<li>libfuse-dev</li>
<li>libssl-dev</li>
<li>build-essential</li>
<li>dh-make</li>
<li>debhelper</li>
<li>devscripts</li>
<li>reprepro</li>
</ul>
</td>
</tr>
</table>
<subsection name="Continuous Integration"></subsection>
<p>
Apache Bigtop uses Jenkins for continuous integration testing (you can learn more about Jenkins
<a href="http://jenkins-ci.org/">here</a>
).
There are 2 Jenkins servers currently running for the project:
</p>
<ul>
<li>
<a href="https://builds.apache.org/view/A-F/view/Bigtop/">https://builds.apache.org/view/A-F/view/Bigtop/</a>
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/">http://bigtop01.cloudera.org:8080/</a>
</li>
</ul>
<p>
The Apache Jenkins server runs the "Bigtop-trunk" job whenever code is committed to the trunk branch, which in turn
runs several test jobs.
</p>
<p>
The Bigtop Jenkins server runs daily jobs for the 0.3 and trunk branches, and each job has a configuration for each
supported operating system.
In each branch there is a job to build each component, a job to run package tests and a job to build package
repositories.
There are also jobs for building virtual machine images as well as the 2 most recent releases and other miscellaneous
tasks for various RCs and tests.
For example:
</p>
<ul>
<li>
<a href="http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-All/">Bigtop-trunk-All</a>
: Runs all of the jobs for the trunk branch.
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-HBase/">Bigtop-trunk-HBase</a>
: Builds the HBase packages only.
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packagetest/">Bigtop-trunk-packagetest</a>
: Runs the package tests.
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-repository/">Bigtop-trunk-repository</a>
: Builds an APT or YUM package repository.
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-0.5.0/">Bigtop-0.5.0</a>
: Builds the 0.5.0 release.
</li>
<li>
<a href="http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/">Bigtop-VM-matrix</a>
: Builds virtual machines of branches trunk and 0.3 for KVM, VMWare and VirtualBox.
</li>
</ul>
</section>
</body>
</document>
|
