Technology plays a vital role in enhancing lives, especially for people with disabilities. Gesture and activity recognition using embedded systems can restore communication, mobility and independence. Yet, enabling such intelligence in ultra resource-constrained sensor systems demands highly efficient, low-power solutions. One promising approach is the use of neural networks with binary or ternary parameters, which significantly reduce memory requirements and computational load. In this paper, we present the training and deployment of binary and ternary neural networks for gesture and human activity recognition on off-the-shelf, ultra resource-constrained sensor systems. The proposed method introduces a fully differentiable, end-to-end framework for training and deploying mixed binary and ternary models directly on the target system. The proposed models offer broad applicability due to their minimal memory requirements and low energy consumption, as validated through experimental evaluation. Experimental results show that the proposed models reduce memory usage by over 80 % and latency by up to 32 %, even when running at a clock speed 5 x slower than state-of-the-art systems.
