TD3 vs. DDPG for Coverage-Driven Beamforming Optimization in RIS-Assisted 6G IoT Systems

Authors

DOI:

https://doi.org/10.69955/ajoeee.2026.v6i1.84

Keywords:

KEYWORDS Reconfigurable Intelligent Surfaces (RIS); Coverage Enhancement; 6G Internet of Things (6G IoT); Deep Deterministic Policy Gradient (DDPG); Deep Reinforcement Learning (DRL); Beamforming Optimization ;

Abstract

Abstract: Reconfigurable intelligent surfaces (RIS) will be necessary in forthcoming 6G mobile networks to ensure flexible coverage. This study presents a simulation model employing Deep Reinforcement Learning to enhance beamforming in RIS-supported systems. We put TD3 and DDPG through 30 different tests, each with a different channel setup. The findings indicate that TD3 exhibits a superior average error rate (13.34 ± 0.73) compared to DDPG (11.13 ± 0.87), reflecting a 19.86% enhancement. This statistical analysis demonstrates that this increase is significant (t ≈ 3.99, p < 0.001) and exhibits a considerable effect size (Cohen's d = 1.03). The average difference is between 1.10 and 3.32 with a 95% confidence level. The variance analysis shows that TD3 has a more stable reward and less variability in performance. The results show that TD3 is a statistically valid way to make smart wireless 6G environments better. This means that it makes IoT connectivity with RIS more reliable and stable.

Downloads

Download data is not yet available.

Author Biographies

  • Ezdihar Osman Taj Almowla Mohomad, University of Bahri

     

    Ezdihar Osman Taj Almowla Mohomad is a Ph.D. Candidate in the Department of Electrical and Electronic Engineering at the University of Bahri, Khartoum, Sudan. She is a lecturer at the University of Hail for a duration of 4 years (January 2022 to the present time)

     

     

  • Khalid Hamid Bilal, University of Bahri

     

    Prof  Khalid Hamid Bilal works in the  Department of Electrical and Electronic Engineering. at the University of Science & Technology. Omdurman. Sudan

  • ZeinabMahmoud Omer, University of Bahri

     

    Dr ZeinabMahmoud Omer works in the  Department of Electrical and Electronic Engineering. at the University of Bahri, Khartoum.

  • Abeer Mohamed Elzain, University of Ha'il

    Abeer Mohamed Elzain  works in the  Electronics & Telecommunication Engineering Technology 

    at the University of Hail , saudi Aradia

     

     

  • Rania Ali Elkhidir

     Dr.Rania Ali Elkhidir is currently working as Assistant Professor in the Department of Information technology at Faculty of Computer Science, Hail University, and a Researcher with an AI Research Group, She was a head of Information System Department in Northern Border University, Rafha Branch , She has experience in Computer Maintenance, Computer Programming , Web Design , She is guiding the Research Students in the area of Information System and Computer Science . Her research interests include text processing, Intelligent Agent , machine learning and deep learning .

References

[ 1] M. Ahmed et al., ‘Toward a Sustainable Low-Altitude Economy: A Survey of Energy-Efficient RIS–UAV Networks’, IEEE Internet of Things Journal, vol. 12, no. 24, pp. 51951–51975, Dec. 2025, doi: 10.1109/JIOT.2025.3618483. DOI: https://doi.org/10.1109/JIOT.2025.3618483

[ 2] Y. Xie, Z. Lin, R. Ma, K. An, X. Zhong, and Y. He, ‘RIS-Empowered Satellite IoT: Bridging the Coverage-Efficiency Gap of Last-Mile Access and Sensing’, IEEE Internet of Things Magazine, pp. 1–8, 2026, doi: 10.1109/MIOT.2026.3658612. DOI: https://doi.org/10.1109/MIOT.2026.3658612

[ 3] S. Zappia, I. Iudice, D. Pascarella, and A. Vozella, ‘UAV-RIS Backscatter IoT Networks: System Models and Performance Analysis’, IEEE Access, vol. 14, pp. 18476–18490, 2026, doi: 10.1109/ACCESS.2026.3658099. DOI: https://doi.org/10.1109/ACCESS.2026.3658099

[ 4] H. Taherdoost, ‘Security and Internet of Things: Benefits, Challenges, and Future Perspectives’, Electronics, vol. 12, no. 8, Apr. 2023, doi: 10.3390/electronics12081901. DOI: https://doi.org/10.3390/electronics12081901

[ 5] K. Joshi, H. Yadav, S. Gupta, V. Singh, K. S. Sidhu, and R. Kukreti, ‘Handling Security Aspects in the Internet of Things: Latest Challenges and Measures to Mitigate Risks’, in 2025 3rd International Conference on Communication, Security, and Artificial Intelligence (ICCSAI), Apr. 2025, pp. 1434–1439. doi: 10.1109/ICCSAI64074.2025.11064678. DOI: https://doi.org/10.1109/ICCSAI64074.2025.11064678

[ 6] K. K. Thangadorai, K. M. Sivalingam, A. Pandey, K. Murugesan, and M. R. Kanagarathinam, ‘WiLongH: A Custom Hand-Held Platform for Long-Range HaLow Mesh Networks in Human-to-Human Communication’, IEEE Open Journal of the Communications Society, vol. 6, pp. 1873–1894, 2025, doi: 10.1109/OJCOMS.2025.3547615. DOI: https://doi.org/10.1109/OJCOMS.2025.3547615

[ 7] ‘Touch in Human Social Robot Interaction: Systematic Literature Review with PRISMA Method | International Journal of Social Robotics | Springer Nature Link’. Accessed: Jan. 22, 2026. [Online]. Available: https://link.springer.com/article/10.1007/s12369-025-01319-1

[ 8] ‘Comparative analysis of Mpox clades: epidemiology, transmission dynamics, and detection strategies | BMC Infectious Diseases | Springer Nature Link’. Accessed: Jan. 22, 2026. [Online]. Available: https://link.springer.com/article/10.1186/s12879-025-11784-8

[ 9] N. Parveen, K. Abdullah, K. Badron, Y. Javed, and Z. I. Khan, ‘Coexistence in Wireless Networks: Challenges and Opportunities’, Telecom, vol. 6, no. 2, Apr. 2025, doi: 10.3390/telecom6020023. DOI: https://doi.org/10.3390/telecom6020023

[ 10] D. G. Arnaoutoglou, T. M. Empliouk, T. N. F. Kaifas, C. L. Zekios, and G. A. Kyriacou, ‘Perspectives and Research Challenges in Wireless Communications Hardware for the Future Internet and Its Applications Services’, Future Internet, vol. 17, no. 6, May 2025, doi: 10.3390/fi17060249. DOI: https://doi.org/10.3390/fi17060249

[ 11] G. A. Akpakwu, T. E. Mathonsi, T. M. Tshilongamulenzhe, S. P. Maswikaneng, and T. Muchenje, ‘Congestion Control in Constrained Application Protocol for the Internet of Things: State-of-the-Art, Challenges, and Future Directions’, IEEE Access, vol. 13, pp. 33733–33767, 2025, doi: 10.1109/ACCESS.2025.3543415. DOI: https://doi.org/10.1109/ACCESS.2025.3543415

[ 12] F. Xiao, Z. Li, and D. Slock, ‘Multipath Component Power Delay Profile Based Joint Range and Doppler Estimation for AFDM-ISAC Systems’, arXiv.org. Accessed: Jan. 22, 2026. [Online]. Available: https://arxiv.org/abs/2503.10833v1

[ 13] ‘Joint Beamforming and Intelligent Reflecting Surface Optimization for Enhanced Physical Layer Security’. Accessed: Jan. 22, 2026. [Online]. Available: https://www.sciopen.com/article/10.26599/TST.2026.9010007

[ 14] ‘A comprehensive survey on reconfigurable intelligent surfaces (RIS) and STAR-RIS for next-generation wireless networks | Discover Applied Sciences | Springer Nature Link’. Accessed: Jan. 22, 2026. [Online]. Available: https://link.springer.com/article/10.1007/s42452-025-07684-w

[ 15] J. An, M. Debbah, T. J. Cui, Z. N. Chen, and C. Yuen, ‘Emerging Technologies in Intelligent Metasurfaces: Shaping the Future of Wireless Communications’, IEEE Transactions on Antennas and Propagation, pp. 1–1, 2025, doi: 10.1109/TAP.2025.3571069. DOI: https://doi.org/10.1109/TAP.2025.3571069

[ 16] H. Jie et al., ‘A review of intentional electromagnetic interference in power electronics: Conducted and radiated susceptibility’, IET Power Electronics, vol. 17, no. 12, pp. 1487–1506, 2024, doi: 10.1049/pel2.12685. DOI: https://doi.org/10.1049/pel2.12685

[ 17] W. Khalid, M. A. U. Rehman, T. Van Chien, Z. Kaleem, H. Lee, and H. Yu, ‘Reconfigurable Intelligent Surface for Physical Layer Security in 6G-IoT: Designs, Issues, and Advances’, IEEE Internet of Things Journal, vol. 11, no. 2, pp. 3599–3613, Jan. 2024, doi: 10.1109/JIOT.2023.3297241. DOI: https://doi.org/10.1109/JIOT.2023.3297241

[ 18] ‘Low Power but High Energy: The Looming Costs of Billions of Smart Devices | ACM SIGEnergy Energy Informatics Review’. Accessed: Jan. 22, 2026. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/3630614.3630617

[ 19] N. H. Trung and N. T. Anh, ‘Beamforming-as-a-Service for Multicast and Broadcast Services in 5G Systems and Beyond’, IEEE Access, vol. 11, pp. 142794–142815, 2023, doi: 10.1109/ACCESS.2023.3343523. DOI: https://doi.org/10.1109/ACCESS.2023.3343523

[ 20] E. Basar et al., ‘Reconfigurable Intelligent Surfaces for 6G: Emerging Hardware Architectures, Applications, and Open Challenges’, IEEE Vehicular Technology Magazine, vol. 19, no. 3, pp. 27–47, Sept. 2024, doi: 10.1109/MVT.2024.3415570. DOI: https://doi.org/10.1109/MVT.2024.3415570

[ 21] Y. Xu, H. Xie, D. Li, and R. Q. Hu, ‘Energy-Efficient Beamforming for Heterogeneous Industrial IoT Networks With Phase and Distortion Noises’, IEEE Transactions on Industrial Informatics, vol. 18, no. 11, pp. 7423–7434, Nov. 2022, doi: 10.1109/TII.2022.3158612. DOI: https://doi.org/10.1109/TII.2022.3158612

[ 22] G. Zhang, D. Zhang, Y. He, J. Chen, F. Zhou, and Y. Chen, ‘Multi-Person Passive WiFi Indoor Localization With Intelligent Reflecting Surface’, IEEE Transactions on Wireless Communications, vol. 22, no. 10, pp. 6534–6546, Oct. 2023, doi: 10.1109/TWC.2023.3244369. DOI: https://doi.org/10.1109/TWC.2023.3244369

[ 23] N. Agrawal, A. Bansal, K. Singh, C.-P. Li, and S. Mumtaz, ‘Finite Block Length Analysis of RIS-Assisted UAV-Based Multiuser IoT Communication System With Non-Linear EH’, IEEE Transactions on Communications, vol. 70, no. 5, pp. 3542–3557, May 2022, doi: 10.1109/TCOMM.2022.3162249. DOI: https://doi.org/10.1109/TCOMM.2022.3162249

[ 24] A. Al-Shafei, H. Zareipour, and Y. Cao, ‘A Review of High-Performance Computing and Parallel Techniques Applied to Power Systems Optimization’, July 06, 2022, arXiv: arXiv:2207.02388. doi: 10.48550/arXiv.2207.02388.

[ 25] Y. Gao et al., ‘AI-Driven Channel State Information (CSI) Extrapolation for 6G: Current Situations, Challenges and Future Research’, Jan. 01, 2026, arXiv: arXiv:2601.00159. doi: 10.48550/arXiv.2601.00159. DOI: https://doi.org/10.1109/COMST.2026.3652799

[ 26] ‘Reinforcement Learning in Dynamic Environments: Challenges and Future Directions | International Journal of Artificial Intelligence, Data Science, and Machine Learning’. Accessed: Jan. 22, 2026. [Online]. Available: https://ijaidsml.org/index.php/ijaidsml/article/view/5

[ 27] M. M. Salim, S. I. Al-Dharrab, D. B. D. Costa, and A. H. Muqaibel, ‘Cooperative NOMA Meets Emerging Technologies: A Survey for Next-Generation Wireless Networks’, Oct. 27, 2025, arXiv: arXiv:2505.16327. doi: 10.48550/arXiv.2505.16327. DOI: https://doi.org/10.1109/OJCOMS.2025.3626862

[ 28] N. Joshi, I. Budhiraja, D. Garg, S. Garg, B. J. Choi, and M. Alrashoud, “Deep reinforcement learning-based rate enhancement scheme for RIS-assisted mobile users underlaying UAV,” Alexandria Engineering Journal,

[ 29] vol. 91, pp. 1–11, 2024, doi: 10.1016/j.aej.2024.01.039 DOI: https://doi.org/10.1016/j.aej.2024.01.039

[ 30] C. Huang, G. Chen, J. Tang, P. Xiao, and Z. Han, ‘Machine-Learning-Empowered Passive Beamforming and Routing Design for Multi-RIS-Assisted Multihop Networks’, IEEE Internet of Things Journal, vol. 9, no. 24, pp. 25673–25684, Dec. 2022, doi: 10.1109/JIOT.2022.3195543. DOI: https://doi.org/10.1109/JIOT.2022.3195543

[ 31] C. Huang, R. Mo, and C. Yuen, ‘Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning’, IEEE Journal on Selected Areas in Communications, vol. 38, no. 8, pp. 1839–1850, Aug. 2020, doi: 10.1109/JSAC.2020.3000835. DOI: https://doi.org/10.1109/JSAC.2020.3000835

[ 32] K. Long, J. Lin, G. Zhao, Y. Zhou, and Y. Mei, ‘DRL-based Joint Beamforming Design for RIS-assisted mmWave MU-MISO system’, in 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP), Nov. 2022, pp. 1131–1136. doi: 10.1109/WCSP55476.2022.10039335. DOI: https://doi.org/10.1109/WCSP55476.2022.10039335

[ 33] M. Iqbal et al., ‘Twin Delayed Deep Deterministic Policy Gradient for Intelligent Optimization in STAR-RIS-Assisted Wireless Networks’, IEEE Open Journal of the Communications Society, vol. 6, pp. 9696–9713, 2025, doi: 10.1109/OJCOMS.2025.3631341. DOI: https://doi.org/10.1109/OJCOMS.2025.3631341

[ 34] https://www.kaggle.com/datasets/ziya07/6g-iot-intelligent-management-dataset

[ 35] S. Pala, K. Singh, O. Taghizadeh, C. Pan, O. A. Dobre, and T. Q. Duong, ‘Robust and Secure Multi-User STAR-RIS-Aided Communications: Optimization Versus Machine Learning’, IEEE Transactions on Communications, vol. 73, no. 9, pp. 7517–7534, Sep. 2025, doi: 10.1109/TCOMM.2025.3541092. DOI: https://doi.org/10.1109/TCOMM.2025.3541092

[ 36] S. Jain, G. Kumar, A. Markan, and C. M. Markan, ‘Downlink Throughput, SINR & Coverage Analysis in Urban Scenario for LTE & mmWave 5G NR MIMO’, in 2025 10th International Conference on Signal Processing and Communication (ICSC), Feb. 2025, pp. 70–75. doi: 10.1109/ICSC64553.2025.10968903. DOI: https://doi.org/10.1109/ICSC64553.2025.10968903

[ 37] S. Pala, K. Singh, O. Taghizadeh, C. Pan, O. A. Dobre, and T. Q. Duong, ‘Robust and Secure Multi-User STAR-RIS-Aided Communications: Optimization Versus Machine Learning’, IEEE Transactions on Communications, vol. 73, no. 9, pp. 7517–7534, Sep. 2025, doi: 10.1109/TCOMM.2025.3541092. DOI: https://doi.org/10.1109/TCOMM.2025.3541092

[ 38] L. Chen, A. Elzanaty, M. A. Kishk, and Y.-J. Angela Zhang, ‘Joint Coverage and Electromagnetic Field Exposure Analysis in Downlink and Uplink for RIS-Assisted Networks’, IEEE Transactions on Wireless Communications, vol. 24, no. 12, pp. 10594–10612, Dec. 2025, doi: 10.1109/TWC.2025.3580603. DOI: https://doi.org/10.1109/TWC.2025.3580603

[ 39] Y. Zhou, Y. Wu, and W. Xu, ‘Multi-functional reconfigurable intelligent surface for maximizing sum rate in wireless communication systems’, AEU - International Journal of Electronics and Communications, vol. 191, p. 155648, Feb. 2025, doi: 10.1016/j.aeue.2024.155648. DOI: https://doi.org/10.1016/j.aeue.2024.155648

[ 40] Z. Sui, H. Q. Ngo, M. Matthaiou, and L. Hanzo, ‘Performance Analysis and Optimization of STAR-RIS-Aided Cell-Free Massive MIMO Systems Relying on Imperfect Hardware’, IEEE Transactions on Wireless Communications, vol. 24, no. 4, pp. 2925–2939, Apr. 2025, doi: 10.1109/TWC.2025.3526563. DOI: https://doi.org/10.1109/TWC.2025.3526563

Downloads

Published

2026-06-22

Issue

Vol. 6 No. 1 (2026)

Section

Articles

License

Copyright (c) 2026 AlamBiblilo Publishers

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

The Asian Journal of Electrical and Electronic Engineering journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

How to Cite

[1]
“TD3 vs. DDPG for Coverage-Driven Beamforming Optimization in RIS-Assisted 6G IoT Systems”, AJoEEE, vol. 6, no. 1, pp. 25–48, Jun. 2026, doi: 10.69955/ajoeee.2026.v6i1.84.

Similar Articles

1-10 of 11

You may also start an advanced similarity search for this article.