| Age | Commit message (Collapse) | Author |
|---|
|
Add support for ODP_CRYPTO_OP_TYPE_OOP operation type.
In openssl crypto, implement out-of-place operations as combinations of
packet data copies and in-place operations. This is done for simplicity
and to ensure correct semantics for API validation test development.
In other crypto modules, do not support out-of-place operations yet,
but fail gracefully by returning ODP_CRYPTO_SES_ERR_PARAMS error in
session creation if creation of an out-of-place session is attempted.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>
|
|
Add support for ODP_CRYPTO_OP_TYPE_BASIC operation type.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>
|
|
Stop supporting deprecated per-session IVs.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>
|
|
Inline odp_crypto_result() function implementation.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Inline odp_crypto_packet_from_event() and odp_crypto_packet_to_event()
function implementations.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
The Arm crypto library may read up to 16 bytes of data when it is reading
a 12 byte IV. If the IV provided by the caller ends at a page boundary and
the next page is not mapped to readable memory, bad things can happen.
Fix the problem by always copying the user provided IV to a bigger
temporary buffer that is passed to the crypto library.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
|
|
The only supported IV length for AES-GCM is 12 bytes. Define it as
a macro to benefit from constant expressions and to get rid of some
magic numbers.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
|
|
Prefix implementation internal debug macro names with underscore.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Enable usage of implementation internal debug macros in inline API headers
by moving the macros into an inline header. _odp_log_fn and _odp_abort_fn
function pointers are moved out of odp_global_data_ro_t to avoid having to
expose the internal data structure.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Do not allocate a new packet but reuse the input packet when a crypto
operation is requested to allocate the output packet and the input
packet and the session use the same packet pool.
Make the code cleaner by moving output packet allocation and preparation
to its own function from crypto_int().
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
The packet metadata copying check in a crypto operation assumes that the
pool of the output packet is the pool configured in the session, but that
is not necessarily the case when an ODP application passes an explicit
output packet to a crypto operation. In that case the pool of the session
could even be ODP_POOL_INVALID.
Fix the bug by reading destination pool from the output packet.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Use an explicit cast to avoid conversion warnings, when application is
compiled with -Wconversion option.
Signed-off-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Use _ODP_ARRAY_SIZE() helper macro for calculating array sizes.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Remove the unused return value from crypto functions.
Indicate error in crypto result if required IV is totally missing with
deprecated API support. Do not bother setting the error code accurately
since the behaviour is anyway undefined after the API usage error.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Remove the unnecessary crypto_err parse flag from packet header.
odp_packet_has_error() is supposed to indicate parse errors not crypto
processing errors, which are indicated through odp_crypto_result_t.
Remove the flag and its setting in crypto implementations as the flag
is not needed and may cause subtle API spec violations.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Depracate odp_crypto_operation() and the associated data structures
and the completion event. The odp_crypto_operation() function is a bit
problematic since it can be synchronous or asynchronous on per-packet
basis without application having control on it. Uncontrolled sync versus
async behaviour can have adverse effects on packet ordering and
application design in general. The function is also somewhat redundant
with the newer odp_crypto_op() and odp_crypto_op_enq() functions that
provide explicit control on the crypto operation completion behavior.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Anoob Joseph <anoobj@marvell.com>
|
|
Have only one cipher algorithm specific function pointer in the session
instead of separate cipher and auth functions. It fits well AEAD
algorithms like AES-GCM which is the only algorithm currently supported.
Single crypto processing function per session makes sense also for
combinations of individual algorithms that are processed in single pass.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Use a stack buffer instead of a thread local buffer for the tag to
avoid possible overhead of accessing thread-local storage.
Use memcpy() instead of the slower packet data copy functions to copy
tag to/from a packet when packet data is continuous.
Use a compile time constant for tag length since only one tag length
is supported. This improves optimization of memcpy() etc.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Call memcpy() with a constant length when copying AAD to a temporary
buffer. This way the generic memcpy call gets optimized away and
replaced by a couple of loads and stores. Take advantage of the fact
that the implementation currently supports only 8 and 12 byte AAD.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
When a packet is not segmented, perform crypto operation in-place to
avoid copying output data from a temporary buffer back to the packet.
Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
|
|
Replace usage of __int128 and __int128_t with _odp_u128_t.
Signed-off-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Use _odp_u128_t instead of various ad hoc typedefs.
Signed-off-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Remove deprecated fields 'md5_96', 'sha256_128', and 'aes128_gcm' from
odp_crypto_auth_algos_t.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Remove deprecated fields 'aes128_cbc' and 'aes128_gcm' from
odp_crypto_cipher_algos_t.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Remove deprecated crypto authentication algorithms ODP_AUTH_ALG_MD5_96,
ODP_AUTH_ALG_SHA256_128, and ODP_AUTH_ALG_AES128_GCM.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Remove deprecated crypto cipher algorithms ODP_CIPHER_ALG_AES128_CBC and
ODP_CIPHER_ALG_AES128_GCM.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Use a single inlined _odp_packet_copy_md() function for copying metadata
for both packets and segments. This function is assumed to never fail. A
new _odp_packet_copy_md_possible() function may be called beforehand to
check that copying packet metadata is possible.
For packets the implementation has been optimized as the user area may not
have to be always copied.
The previous segment metadata copy function wrote only the destination
segment's user area pointer, which caused both source and destination
segments to use the same area. The old copy function was also missing
classifer mark metadata.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Tuomas Taipale <tuomas.taipale@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Implement AES-GCM in armv8crypto using the AArch64cryptolib.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan <govindarajan.mohandoss@arm.com>
Reviewed-by: Janne Peltonen <janne.peltonen@nokia.com>
|
|
Integrate Aarch64cryptolib in the build using pkg-config. Add an
autoconf config option to select the crypto implementation to be used
and add armv8crypto as an alternative to openssl and null. Add a null
crypto based skeleton crypto implementation to be used with the
Aarch64cryptolib.
Signed-off-by: Govindarajan <govindarajan.mohandoss@arm.com>
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Janne Peltonen <janne.peltonen@nokia.com>
|
|
Implement true random data on aarch64 via RNDR register introduced as an
extension in Armv8.5-a ISA.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan <govindarajan.mohandoss@arm.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Added part number enumerations for ARM V1, N2, A510 and
A710 CPUs.
Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
|
|
CPU specific hardware/software features are exposed by the kernel to the
userspace through a set of flags called hwcaps, exposed in the auxiliary
vector. Userspace software can test for features by acquiring the AT_HWCAP
and AT_HWCAP2 entry of the auxiliary vector.
These hwcaps are defined in terms of architected system register fields
and those exposed in AT_HWCAP and AT_HWCAP2 typically have feature flags
associated with them. These mapped feature flags are printed.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan <govindarajan.mohandoss@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
More compile-time CPU flags have been added to test the support for newer
AArch64 features.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Detection and printing of compile-flags has been moved to a new file,
cpu_flags.c in preparation for dumping the CPU hardware flags.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Fix "ISO C forbids casts to union type" errors when building with
'pedantic' option.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Fix "ISO C forbids braced-groups within expressions" errors when building
with 'pedantic' option.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Fix "ISO C does not support '__int128' types" errors when building with
'pedantic' option.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
GCC generates LDADDL instructions when using __atomic_fetch_add() on
aarch64. Explicitly use STADDL to implement odp_atomic_add_rel_u64() and
odp_atomic_sub_rel_u64() functions for better performance.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Govindarajan Mohandoss <govindarajan.mohandoss@arm.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
GCC generates LDADDL instructions when using __atomic_fetch_add() on
aarch64. Explicitly use STADDL to implement odp_atomic_add_rel_u32() and
odp_atomic_sub_rel_u32() functions for better performance.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Govindarajan Mohandoss <govindarajan.mohandoss@arm.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
By default, GCC generates the LDADD/LDADDL instruction for atomic
add/sub/inc/dec functions. These functions could also be implemented on
ARMv8.1 ISA (and above) with the store version, STADD/STADDL instruction
which result in better performance for fastpath applications.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan Mohandoss <govindarajan.mohandoss@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
Refactor CPU ABI and implementation files to follow common project inlining
style. odp_cpu_pause() function is now defined only once per architecture.
After this clean-up it's simpler for an implementation to add inlined
versions of CPU module functions.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
|
|
Previously, inlined atomic functions (atomic_inlines.h) were not included
through the API headers in non-ABI compat mode on aarch64 platforms.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Implemented strict versions of local and global time stamp read
functions. Moved time register read code into inline functions
on ARMv8.
Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
|
|
Detect CPU cache line size automatically on aarch64 systems when abi-
compatibility is disabled. Automatic cache line size detection is disabled
when cross-compiling.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Added support for ARM ISA versions: ARMv8.7-A, ARMv9.0-A,
ARMv9.1-A and ARMv9.2-A.
Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com>
Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
On arm64 systems, the build fails with the --disable-host-optimization
compiler flag as it is unable to locate definitions of lockless APIs
used in atomic bit operations since these declarations are present in
arch/aarch64/odp_atomic.h files Consequently, the aarch64 specific code
has been moved to the arch-specific file while the reminder code has been
moved under the default linux-generic implementation.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan Mohandoss <govindarajan.mohandoss@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
|
|
On arm64 systems, the build fails with the --disable-host-optimization
compiler flag as it is unable to locate definitions of lockless APIs
present in arch/aarch64/odp_atomic.h files. Consequently, the aarch64
specific code in odp_queue_lf.c has been moved to the arch-specific file
while the reminder code has been added to a new file under the default
linux-generic implementation.
Signed-off-by: Malvika Gupta <Malvika.Gupta@arm.com>
Reviewed-by: Govindarajan Mohandoss <govindarajan.mohandoss@arm.com>
Reviewed-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Optimize arm64 specific 128-bit atomics implementation. The init function
doesn't have to be atomic and CAS operations' 'old_val' needs to be updated
only on failure.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Add new implementation for the 128-bit atomic operations using GCC builtin
atomics. This improves performance considerably compared to the basic
lock-based implementation.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
|
Enables architecture specific implementations.
Signed-off-by: Matias Elo <matias.elo@nokia.com>
Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
|
