| Age | Commit message (Collapse) | Author |
|---|
|
On Python 3.x this has the side-effect of returning a str instead of a bytes.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
In Python 3.x the strerr is more verbose, and thus fails the equals
test.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
It is no longer supported in Python 3.x, but unpacking the tuple parameter
inside the function works just fine on both old and new Python versions.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
The old syntax is no longer supported in Python 3.x.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
The old syntax is no longer supported in Python 3.x.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
These warnings have become fatal errors in Python 3.x.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
|
|
|
|
Move the re.compile into the core.Enivironment constructor, which
reduces code duplication. It also allows us to pass environment data on
initilization of the object, rather that having edit it's attributes
individually.
V2: - Does not remove deprecated options, only marks them as such
V3: - Fixes deperecated warning for tests from V2 always being triggered
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
|
|
Was getting error about an unknown keyword 'wglinfo'.
Signed-off-by: José Fonseca <jfonseca@vmware.com>
|
|
This fixes resume failures when paths are interpreted as unicode when the
script expects ascii.
[chadv]: Replace `if (!X and !Y) assert()` with `assert(X or Y)`.
Reviewed-and-tested-by: Chad Versace <chad.versace@linux.intel.com>
|
|
With commit dadf56dad25730556e36e9446aabfc80458cc12f the syntax
for the [require] section in the shader test data files changed for
GL ES. This patch, updates the shader_runner.py regex to also
understand the new syntax since this code parses the [require]
section of the shader test data file to determine which
shader_runner binary to run.
When dispatch is implemented for GLES all this can go away.
v2: formatting, better description, no GLSL ES, GLSL detection.
Signed-off-by: Tom Gall <tom.gall@linaro.org>
Reviewed-by: Paul Berry <stereotype441@gmail.com>
|
|
Fix shader_test.py so it uses shader_runner_gles2 when running
a shader test with GL 2.0 es in the [required] section.
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
Signed-off-by: Tom Gall <tom.gall@linaro.org>
|
|
Subtests allow a test to produce more than one result. This is useful
for tests where it may be convenient to combine several somewhat
unrelated tests into a single program. For example, a cl test like:
kernel void test(TYPE *out, TYPE *in) {
out[0] = some_builtin_function(in[0], in[1]);
}
That uses only one kernel, but recompiles it several times with different
definitions of the macro 'TYPE' in order to test the builtin with all
the possible types.
To take advantage of the subtest feature, programs should output one
PIGLIT line per subtest as in the following example:
PIGLIT:subtest {'testA' : 'pass'}
PIGLIT:subtest {'testB' : 'fail'}
Where testA and testB are the name of the subtests.
In the result summary, this will be displayed as:
TestName 1/2
testA pass
testB fail
v2:
- Print one line for each subtest rather than printing them all
together.
Acked-by: Chad Versace <chad.versace@linux.intel.com>
|
|
At this point, Glean wasn't writing anything interesting anyway;
it was just clutter.
Since there's no need to specify a results directory on the command line
any longer, this patch also removes the -r option, making "run tests"
the default action.
This also allows us to simplify the Piglit runner framework a little:
it no longer has to pass around the results directory just to pass to
Glean.
|
|
Commit 260f211 caused some shader tests to regress with the message:
"Failed to find [require] block".
This patch changes the python script that drives shader_runner to assume
that the test's API is GL if no GL requirement is found in the test file.
This behavior matches the behavior of the shader_runner executable, whose
default requirements are GL >= 1.0 and GLSL >= 1.10.
I tested this patch with a full Piglit run. The delta against
master-ed3903c consists only of fixes.
Reviewed-by: Tom Gall <tom.gall@linaro.org>
Tested-by: Tom Gall <tom.gall@linaro.org>
Reported-by: Ian Romanick <idr@freedesktop.org>
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
|
|
Add a new class to shader_test.py, ShaderTest. ShaderTest.run() parses the
[require] block of the shader test to determine which shader_runner
executable should be used to run the test.
This is needed because GLES3 shader tests must be run by
shader_runner_gles3 and the GL tests by shader_runner.
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
|
|
This patch is a small refactor. It moves add_shader_test_dir() from
all.tests into a new module, shader_test.py. Follow-on commits will expand
the role of the new module.
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
|
|
Otherwise python2 blows up when the program output contains a non-ascii
character. The resulting exception caused a bunch of spurious fails in
the igt testscases.
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
The intel-gpu-tools testrunner needs this to figure out the
testresult (since 'skip' is indicated by a returncode of 77
in i-g-t).
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
As we do glxinfo on Linux. Note that if wglinfo.exe isn't found
you'll see a line like this in the results file:
"wglinfo": "Failed to run wglinfo",
Reviewed-by: José Fonseca <jfonseca@vmware.com>
|
|
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
glean can be modified to use waffle, and thus support gbm.
But in order to retain the multiple visuals testing for X,
the waffle API will need to be updated. Therefore, for now
we'll just skip glean with gbm.
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
If the PIGLIT_PLATFORM environment variable is set, then read its value
into the python PIGLIT_PLATFORM global variable. If it is not set, then
PIGLIT_PLATFORM is set to an empty string.
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
check_for_skip_scenario allows a test to be marked as 'always skip'
Currently it doesn't mark any test for skipping.
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
Previously, Piglit reported 'fail'.
Different sets of tests are built under different build configurations. If
a developer chooses to not build a test, Piglit shouldn't report failure.
Otherwise, it appears as a regression.
Now that 'returncode' may be None, any string formatting that references
it must be converted from Python2-style (%) to Python3-style (str.format).
Reviewed-and-tested-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Paul Berry <stereotype441@gmail.com>
Acked-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
|
|
Reviewed-by: Brian Paul <brianp@vmware.com>
|
|
Otherwise, we'd see massive spam of WARN tests, due to the frequency
of the bug (about 2% of testcases on my system).
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
|
|
test name will always use '/'.
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
|
|
Otherwise, if the user forgets to set PIGLIT_SOURCE_DIR himself, then
tests that use piglit_source_dir() will fail.
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
|
|
I want to be able to get a list of commands being run without specifying
an actual result directory. So, I'm specifying it as $GLEAN_RESULT_DIR.
In that case, I don't want it to create a literal directory called
$GLEAN_RESULT_DIR. (The % is for python-style substitutions.)
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
The junit format used by Jenkins came from Java land, and has three
levels: packages, classes, and test methods; so mapping a generic test
hierarchy to these is far from straightforward.
This change tries to rearrange the mapping, in such a way that the test
results as presented by Jenkins look better and are easier to navigate.
|
|
While distinguishing between 'fail' and 'crash' in extremely useful,
I've never encountered a situation where 'abort' or 'trap' offers more
insight than simply calling it a 'crash'.
We may as well simplify things a bit.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Acked-by: Marek Olšák <maraeo@gmail.com>
|
|
In other words, if running piglit-run.py tests/all.tests results/foo,
put glean's output in results/foo/glean instead of the more redundant
results/foo/results/glean.
Reviewed-by: Eric Anholt <eric@anholt.net>
|
|
Before scheduling or running the tests, run() prepared the final list of
tests to run: first, flatten the Group() hierarchy; second, filter out
tests based on the -t and -x options.
It makes sense to have this as a helper function. Doing so will also
enable other utilities that (for example) print a list of tests that
would be run and their command line programs.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
|
|
Valgrind testing is useful, but really should be done as a separate
exercise from the usual regression testing, as it takes way too long.
Rather than including it by default in all.tests and making people
exclude it with the -x valgrind option (or by using quick.tests), it
makes sense to explicitly request valgrind testing with --valgrind.
To perform valgrind testing:
$ piglit-run.py --valgrind <options> tests/quick.tests results/vg-1
The ExecTest class now handles Valgrind wrapping natively, rather than
relying on the tests/valgrind-test/valgrind-test shell script wrapper.
This provides a huge benefit: we can leverage the interpretResult()
function to make it work properly for any subclass of ExecTest. The
old shell script only worked for PlainExecTest (piglit) and GleanTest.
Another small benefit is that you can now use --valgrind with any test
profile (such as quick.tests). Also, you can use all.tests without
having to remember to specify "-x valgrind".
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
GPUs like to hang, especially when barraged with lots of mean Piglit
tests. Usually this results in the poor developer having to figure out
what test hung, blacklist it via -x, and start the whole test run over.
This can waste a huge amount of time, especially when many tests hang.
This patch adds the ability to resume a Piglit run where you left off.
The workflow is:
$ piglit-run.py -t foo tests/quick.tests results/foobar-1
<interrupt the test run somehow>
$ piglit-run.py -r -x bad-test results/foobar-1
To accomplish this, piglit-run.py now stores the test profile
(quick.tests) and -t/-x options in the JSON results file so it can tell
what you were originally running. When run with the --resume option, it
re-reads the results file to obtain this information (repairing broken
JSON if necessary), rewrites the existing results, and runs any
remaining tests.
WARNING:
Results files produced after this commit are incompatible with older
piglit-summary-html.py (due to the extra "option" section.)
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Paul Berry <stereotype441@gmail.com>
|
|
When resuming an interrupted piglit run, we'll want to output both
existing and new results into the same 'tests' section. Since
TestProfile.run only handles newly run tests, it can't open/close the
JSON dictionary.
So, move it to the caller in piglit-run.py.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Paul Berry <stereotype441@gmail.com>
|
|
Commit 4fbe147b5817b2ba0fe44fe9db96a2d05288aee introduced a regression
where not specifying a -t option would result in all tests being
excluded.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
del dict[key] will raise a KeyError if the key isn't actually in the
dictionary. This can happen if, say, a test didn't match the -t option
and subsequently matched a -x option.
Also, deleting dictionary items while iterating over them is apparently
not safe in Python 3, so we may as well just create a new dictionary
with only the entries we want.
This is simpler anyway.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
If by some bizarre circumstance, self.command were to be unicode, this
would fail because type(self.command) wouldn't be types.StringType.
Also, I don't think shlex.split handles unicode, so just convert it.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Paul Berry <stereotype441@gmail.com>
|
|
It was polite to print a message shortly after the conversion, but at
this point, it's basically dead code.
|
|
all.tests includes a series of regular expressions to discard driver
chatter that Piglit shouldn't consider a warning. Unfortunately, it
got copy and pasted to a few more files.
Move it back into one place---in core. While we're at it, use `map' to
avoid having to write Test.ignoreErrors.append(re.compile(...)) every
time.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
Because seriously, who wants to sleep between tests? Under normal
circumstances, this just pointlessly slows test runs down. During
platform bring-up, or when debugging flushing issues, it can be useful,
but it's just as easy to hack in a sleep(0.2) call yourself.
|
|
It doesn't make much sense for Tests to filter themselves out based on
global options just before they're about to run; they should be filtered
out ahead of time. This avoids scheduling a bunch of useless tests.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
Test.doRun was serving two purposes:
1. Queueing up a concurrent test for later execution.
2. Running a test immediately (in the current thread).
This behavior was controlled by an effectively global parameter,
Environment.run_concurrent, which could easily be confused with
Test.runConcurrent (whether the test is safe to run concurrently) and
Environment.concurrent (whether to use multiple threads/the -c flag).
Split it into two functions for clarity.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
In the past, Piglit has stored test profiles as a hierarchy of Test and
Group objects: TestProfile contained a top-level Group, and each Group
could contain either Tests or Groups. To refer to a test, one could do:
tests['spec']['glsl-1.30']['preprocessor']['compiler']['keywords']['void.frag']
A second way of referring to tests is via their "fully qualified name",
using slashes to delimit groups. The above example would be:
tests['spec/glsl-1.30/preprocessor/compiler/keywords/void.frag']
This fully qualified name is used for regular expression matching
(piglit-run.py's -t and -x options). Since the advent of JSON, it's
also what gets stored in the results file.
Using the Group hierarchy is both complicated and inconvenient:
1. When running tests, we have to construct the fully qualified
name in order to write it to the JSON (rather than just having it).
2. Adding the ability to "resume" a test run by re-reading the JSON
results file and skipping tests that have already completed would
require the opposite conversion (qualified name -> hierarchy).
3. Every Group has to be manually instantiated and added to the parent
group (rather than just being created as necessary).
4. The Group class is actually just a dumb wrapper around Python's dict
class, but with extra inconvenience.
This patch is an incremental step toward dropping the Group hierarchy.
It converts TestProfile.run() to use a simple dictionary where the keys
are fully qualified test names and the values are the Test objects. To
avoid having to convert all.tests right away, it converts the Group
hierarchy into this data structure before running.
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
|
|
To avoid interfering w/ the test hierarchy, which is determined by the
dots.
|
