What is Dense Tensor?
A standard multidimensional array storing all elements in memory, providing fast sequential access but consuming significant storage at large scale
Summary
- Stores every element explicitly in contiguous memory blocks, making them efficient for standard matrix operations, convolutions, and algorithms optimized for sequential access patterns
- Well-suited for dense data like images, audio signals, and fully-connected neural network layers where most values are non-zero and computation involves all elements
- Default tensor representation in frameworks like TensorFlow, PyTorch, and NumPy, leveraging hardware optimizations for dense linear algebra operations on CPUs, GPUs, and TPUs
Dense tensors store values in a contiguous sequential block of memory where all values are represented. Tensors or multi-dimensional arrays are used in a diverse set of multi-dimensional data analysis applications. There are a number of software products that can perform tensor computations, such as the MATLAB suite that has even been enhanced by various open source third party toolboxes. MATLAB alone is capable of supporting a variety of element-wise and binary dense tensor operations A dense layer is a fully connected layer, as each and every neuron gets an input from all the neurons in the previous layer, thus being densely connected. This means that every Neuron in a Dense layer will be fully connected to every Neuron in the prior layer. 
Dense is usually used towards the end of a network, and sometimes multiple times. Trying to build a layered infrastructure for high-performance dense tensor applications, one of the most used libraries is dten, which is known for storing and manipulating dense tensors. The library focuses on storing dense tensors in canonical storage formats and converting between storage formats in parallel. In addition, it supports tensor matricization in different ways. The library is general-purpose and provides a high degree of flexibility. We may regard a tensor as the multidimensional generalization of a matrix. Mathematically, matricization is merely a conceptual (or logical) restructuring of the tensor.
