aboutsummaryrefslogtreecommitdiff
path: root/platform/linux-generic/odp_pool.c
AgeCommit message (Collapse)Author
2021-02-26linux-gen: rename remaining global symbols in the static libraryJere Leppänen
Rename the rest of the global symbols in the static library that do not have the "_odp_" prefix. Fixes: https://github.com/OpenDataPlane/odp/issues/805 Signed-off-by: Jere Leppänen <jere.leppanen@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2021-02-05linux-gen: packet: ensure header size with static assertPetri Savolainen
Ensure that buffer and packet header sizes do not accidentally grow to new cache lines when adding new struct fields. Removed comments about data alignment as those may not be always valid (e.g. 32 bit builds). Added debug prints for checking data alignment on the same offsets. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2021-01-15linux-gen: pool: add maximum packet length config optionMatias Elo
Add configuration file option for the maximum packet data length in bytes. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
2020-12-23linux-gen: pool: remove old debug pool statisticsMatias Elo
Old implementation specific pool statistics can be removed now when the pool API provides similar functionality. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
2020-12-23linux-gen: pool: implement support for pool statisticsMatias Elo
Implement new odp_pool_stats_reset() and odp_pool_stats() functions. To minimize overhead, support for alloc_ops, free_ops, cache_alloc_ops, and cache_free_ops counters can be disabled by setting CONFIG_POOL_STATISTICS define (platform/linux-generic/include/odp_config_internal.h) to zero. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
2020-12-23linux-gen: pool: implement new pool index functionsMatias Elo
Implement new odp_pool_index() and odp_pool_max_index() functions. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Jere Leppänen <jere.leppanen@nokia.com>
2020-11-20linux-gen: pool: update odp_buffer_is_valid() implementationPetri Savolainen
Update odp_buffer_is_valid() to conform API spec change. Try to avoid crash with corrupted buffer handles by checking that buffer header pointer is on a valid range before accessing buffer header fields. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2020-11-19linux-gen: pool: implement vector pool typeMatias Elo
Implementation for the new vector pool type. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2020-11-16linux-gen: unify ODP_ALIGNED usageMatias Elo
Use ODP_ALIGNED (GCC's aligned attribute) according to the GCC documentation. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2020-06-11linux-gen: pool: fix overflow when creating a huge packet poolJanne Peltonen
Convert multiplication operands, not the result, to uint64_t to avoid overflow when calculating the number of extra buffers for skipping huge page boundaries. An overflow would have required a pool size of gigabytes. Signed-off-by: Janne Peltonen <janne.peltonen@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2020-06-05linux-gen: pool: include pool index into SHM namePetri Savolainen
Format pool and uarea SHM names to be similar and include pool index into the names. This helps debugging as pool names are optional or may not be unique. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com>
2020-06-05linux-gen: pool: per thread counters for debuggingPetri Savolainen
Added per thread counters for total number of events allocated and freed (per pool). These are enabled when debugging (requires --enable-debug). Results are printed on thread terminate (requires --enable-debug-print). Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com>
2020-05-15linux-gen: pool: implement cache size parameterPetri Savolainen
Implement new cache size parameters. Cache and burst sizes are now defined per pool. Cache size value from config file is used to initialize cache size parameters. Burst size value from config file is used, when cache size is larger than 2x burst size. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2020-05-15api: add missing const qualifiersPetri Savolainen
Some API calls missed const qualifiers on read-only data pointers. Added those and fixed implementation as well to avoid build errors. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Stanislaw Kardach <skardach@marvell.com>
2020-01-28linux-gen: pool: set packet pool max align capaPetri Savolainen
Implement the new packet align pool parameter with minimal changes. Packet pools are aligned always to the base align, so all application requests up to that value are supported. Base align can be changed through the config file. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Stanislaw Kardach <skardach@marvell.com>
2019-11-26linux-gen: pool: check global pointer on terminatePetri Savolainen
Global pointer is NULL, if init fails. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-11-26linux-gen: pool: use align config optionsPetri Savolainen
Setup alignment according to config file options. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-11-26linux-gen: pool: add data align config file optionsPetri Savolainen
Added packet and buffer data alignment options into the config file. For packets, this is the segment base address alignment. For buffers, this is minimum data alignment, application alignment requirements are rounded up to this value. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-11-26linux-gen: pool: rename pool global variablePetri Savolainen
Renamed global variable for better code readability. Use _odp prefix as the name is visible to application when APIs are inlined. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-11-18linux-gen: pool: decrease number of pools capaPetri Savolainen
At least one pool needs to be reserved from ODP_CONFIG_POOLS for internal usage. Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-11-18linux-gen: pool: add line feed to error printsPetri Savolainen
Signed-off-by: Petri Savolainen <petri.savolainen@nokia.com> Reviewed-by: Matias Elo <matias.elo@nokia.com>
2019-10-23linux-gen: packet: reimplement segmentation using linked listMatias Elo
Packet segmentation has been reimplemented using a linked list. The new implementation is more simple and has the added benefit of reducing packet and buffer header sizes: odp_packet_hdr_t: 384B -> 256B odp_buffer_hdr_t: 256B -> 64B The dynamic packet reference implementation has been temporarily downgraded to a simple packet copy. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-10-22linux-gen: init: prefix internal init/term functions with underscoreMatias Elo
Prefix internal init/term functions with underscore to improve code readability. Unused odp_timer_disarm_all() function declaration is removed. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-09-27linux-gen: include config.h header in install directoryMatias Elo
Previously, config.h header was not included in install dir. This caused a compilation failure when an application tried to include ODP helper headers. config.h has been moved to include/odp subdirectory to avoid naming conflicts and unnecessary includes have been removed. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com> Reported-by: Mikko Parpala <mikko.parpala@nokia.com>
2019-08-16linux-gen: dpdk: remove unnecessary zero-copy block size round upMatias Elo
The blocks size is already rounded up by rte_mempool_calc_obj_size(). Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-04-17linux-gen: pool: use pointer ring to store buffer headersMatias Elo
Modify pool implementation to store buffer headers instead of indices. This removes the need for pointer <-> index conversion in buffer alloc/free functions. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-04-17linux-gen: ring: enable storing pointers in ring dataMatias Elo
Added new ring type ring_ptr_t, which uses the same implementation as old ring_t, but stores pointers instead of indices. The old ring_t type is renamed to ring_u32_t. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-04-15linux-gen: pool: warn about entire pool fitting into thread local cachesMatias Elo
Add debug print when entire pool fits into thread local caches. Pool starvation may occur if the pool is used by multiple threads. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com> Suggested-by: Mikko Kurikka <mikko.kurikka@nokia.com>
2019-04-15linux-gen: pool: enable setting local cache and burst size using configMatias Elo
Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-04-15linux-gen: pool: optimize buffer cachingMatias Elo
Refactor local buffer caching into separate functions and optimize implementation by caching buffer header pointers instead of indices. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2019-03-18linux-gen: shm: implement new ODP_SHM_HP flagMatias Elo
Internal _ODP_ISHM_USE_HP flag is replaced with API ODP_SHM_HP flag. This makes internal _odp_shm_reserve() wrapper function unnecessary. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Stanislaw Kardach <skardach@marvell.com> Reviewed-by: Petri Savolainen <petri.savolainen@nokia.com>
2018-11-30linux-gen: pool: remove unnecessary align padding from buffersMatias Elo
Buffer and timeout header sizes are rounded up to cache line size, so the following data can be cache line aligned without extra padding. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-11-30linux-gen: pool: move ODP_CONFIG_BUFFER_ALIGN_MIN out of config headerMatias Elo
Rename ODP_CONFIG_BUFFER_ALIGN_MIN to BUFFER_ALIGN_MIN and move it to odp_pool.c as this value links closely to the buffer pool implementation. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-11-30linux-gen: pool: reduce buffer memory usageMatias Elo
Calculate memory block size using pool type specific header. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-11-14linux-gen: ishm: add internal _ODP_ISHM_USE_HP flagMatias Elo
Add internal shm flag for allocating shm memory always from huge pages. This is required by zero-copy dpdk packet pool. Internal _odp_shm_reserve() function is added for passing extra shm flags. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-11-14linux-gen: ishm: remove _ODP_SHM_NO_HP flagMatias Elo
Reserving large shm blocks without huge pages increases page misses. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-18linux-gen: pool: add max num packets in config filePetri Savolainen
This config is used to for maximum capability. The default capability needs to be modest so that system memory limit is not exceeded. User may increase maximum number of packets when system memory size allows (and SHM single VA is not used). Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-18linux-gen: pool: add packet param checksPetri Savolainen
Do also checks against maximum number of packets and max packet headroom size. Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-18linux-gen: pool: output error on pool createPetri Savolainen
Change from debug to error message on pool create checks. These are slow path errors and it's helpful to see those also when not debugging. Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-15linux-gen: pool: ring size must be larger than num itemsPetri Savolainen
Ensure that ring size is larger than number of events to be stored in there. Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-11linux-gen: dpdk: improved zero-copy implementationMatias Elo
Improved zero-copy DPDK pktio implementation which better adheres to DPDK APIs. The new implementation reduces overhead by moving mbuf initialization to ODP pool create and by using offsets instead of saved pointers to do ODP packet / DPDK mbuf conversion. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-10-09linux-gen: do not use huge pages for internal allocationsMaxim Uvarov
Some linux-generic internal shared memory allocations have to be in normal pages due to small required data. Relaying on odp_sys_huge_page_size() is not really correct, because call returns default huge page size. But default huge page size is definned in kernel boot parameter as: default_hugepagesz=1G hugepagesz=1G hugepages=2 So in that case for small allocation linux-gen will create 1GB huge page. This patch introduces internal flag to shm funtion to allocate hp. This patch remains allowing changes on top of it: For now pools are in huge page. And for apps with small pool it's big overhead of unused memory. We should take into account odp_sys_huge_page_size_all() call and found best sized huge pages. https://bugs.linaro.org/show_bug.cgi?id=3954 Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org>
2018-10-04linux-gen: split global data into RO and RW structsMatias Elo
RW global data is allocated from shm to enable process mode support. Signed-off-by: Matias Elo <matias.elo@nokia.com> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-09-28linux-gen: shm: reserve internal shms using single VA flagMatias Elo
Reserve internal shm blocks using single VA flag when the reservation is done after global init. Enables process mode support. Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-08-10abi: pool: set ODP_POOL_INVALID to zeroMatias Elo
Signed-off-by: Matias Elo <matias.elo@nokia.com> Reviewed-by: Petri Savolainen <petri.savolainen@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-05-28linux-gen: init: split odp_internal.hPetri Savolainen
odp_internal.h contained code for three different purposes. Split file into three different headers, so that it's no more a place for random type/function prototype additions. New files are: * odp_init_internal.h which holds global/local init function prototypes * odp_global_data.h which defines global data structure * odp_sysinfo_internal.h which contains system info internal function prototypes Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-05-22linux-gen: use inlined thread functions everywhereDmitry Eremin-Solenikov
Make linux-gen platform code use inlined thread functions instead of doing proper functions calls. Signed-off-by: Dmitry Eremin-Solenikov <dmitry.ereminsolenikov@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-05-21linux-gen: ticketlock: remove separate API inlining headerDmitry Eremin-Solenikov
Signed-off-by: Dmitry Eremin-Solenikov <dmitry.ereminsolenikov@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-05-10linux-gen: buffer: remove buffer_inlines header filePetri Savolainen
Moved odp_buffer_inlines.h content to odp_buffer_internal.h and removed the file. Other xxx_inlines.h headers do not contain internal functions but only code for API function inlining. Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Reviewed-by: Dmitry Eremin-Solenikov <dmitry.ereminsolenikov@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
2018-04-17linux-gen: fix clang on aarch64 buildMaxim Uvarov
No code change. Just move array definition several lines up to make clang happy to generate valid code. Fixes: https://bugs.linaro.org/show_bug.cgi?id=3611 Suggested-by: Dmitry Eremin-Solenikov <dmitry.ereminsolenikov@linaro.org> Reviewed-by: Bill Fischofer <bill.fischofer@linaro.org> Reviewed-by: Petri Savolainen <petri.savolainen@linaro.org> Reviewed-by: Dmitry Eremin-Solenikov <dmitry.ereminsolenikov@linaro.org> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-16 21:45:51 +0000