Browsing by Author "Zhou, Sheng"

Now showing 1 - 2 of 2
  • Thumbnail Image
    SMART: Situationally-Aware Multi-Agent Reinforcement Learning-Based Transmissions
    Jiang, Zhiyuan; Liu, Yan; Hribar, Jernej; DaSilva, Luiz A.; Zhou, Sheng; Niu, Zhisheng (IEEE, 2021-12-01)
    In future wireless systems, latency of information needs to be minimized to satisfy the requirements of many mission-critical applications. Meanwhile, not all terminals carry equally-urgent packets given their distinct situations, e.g., status freshness. Leveraging this feature, we propose an on-demand Medium Access Control (MAC) scheme, whereby each terminal transmits with dynamically adjusted aggressiveness based on its situations which are modeled as Markov states. A Multi-Agent Reinforcement Learning (MARL) framework is utilized and each agent is trained with a Deep Deterministic Policy Gradient (DDPG) network. A notorious issue for MARL is slow and non-scalable convergence – to address this, a new Situationally-aware MARL-based Transmissions (SMART) scheme is proposed. It is shown that SMART can significantly shorten the convergence time and the converged performance is also dramatically improved compared with state-of-the-art DDPG-based MARL schemes, at the expense of an additional offline training stage. SMART also outperforms conventional MAC schemes significantly, e.g., Carrier Sensing and Multiple Access (CSMA), in terms of average and peak Age of Information (AoI). In addition, SMART also has the advantage of versatility – different Quality-of-Service (QoS) metrics and hence various state space definitions are tested in extensive simulations, where SMART shows robustness and scalability in all considered scenarios.
  • Thumbnail Image
    Timely and sustainable: Utilising correlation in status updates of battery-powered and energy-harvesting sensors using Deep Reinforcement Learning
    Hribar, Jernej; DaSilva, Luiz A.; Zhou, Sheng; Jiang, Zhiyuan; Dusparic, Ivana (Elsevier, 2022-08-01)
    In a system with energy-constrained sensors, each transmitted observation comes at a price. The price is the energy the sensor expends to obtain and send a new measurement. The system has to ensure that sensors' updates are timely, i.e., their updates represent the observed phenomenon accurately, enabling services to make informed decisions based on the information provided. If there are multiple sensors observing the same physical phenomenon, it is likely that their measurements are correlated in time and space. To take advantage of this correlation to reduce the energy use of sensors, in this paper we consider a system in which a gateway sets the intervals at which each sensor broadcasts its readings. We consider the presence of battery-powered sensors as well as sensors that rely on Energy Harvesting (EH) to replenish their energy. We propose a Deep Reinforcement Learning (DRL)-based scheduling mechanism that learns the appropriate update interval for each sensor, by considering the timeliness of the information collected measured through the Age of Information (AoI) metric, the spatial and temporal correlation between readings, and the energy capabilities of each sensor. We show that our proposed scheduler can achieve near-optimal performance in terms of the expected network lifetime.