DRL-Aided Joint Resource Block and Beamforming Management for Cellular-Connected UAVs
Accepted by IEEE Global Communications Conference (GLOBECOM), Aug. 2023.
To be updated soon enough.
Publications
- Please note that I am the first-order author for all publications illustrated on this page. A more comprehensive track record covering all my authored papers can be found on my Google Scholar page.
- Quick facts: I, as the first-order author, published 3 Transactions (2 IEEE TWC, 1 IEEE TCom), 1 Letter (IEEE WCL) and 4 Conferences (2 IEEE ICC, 1 IEEE PIMRC and 1 IEEE GLOBECOM) around my PhD graduation. I thus have gained extensive knowledge of Python (Torch, TensorFlow, etc.) and Matlab.
2023
Secrecy Performance Analysis on UAV Down-Link Broadcasting with a Full Duplex Receiver
Accepted by IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Jun. 2023.
In this paper, physical layer security issue for a down-link wireless communication system is examined, composed of an unmanned aerial vehicle (UAV), a legitimate receiver and a passive eavesdropper. The destination is equipped with two antennas and applies the full-duplex (FD) Bob-based jamming (FD-BBJ) strategy to achieve secure transmission. Considering that practical air-to-ground (A2G) channels experience Nakagami-$m$ fading and the FD legitimate receiver is affected by self-interference (SI), closed-form expressions of approximate ergodic achievable secrecy rate (EASR) with help of Gauss-Laguerre Quadrature (GLQ) and compact secrecy outage probability (SOP) expression are derived, respectively. To gain more insights, asymptotic secrecy performance is analysed in the case of extreme total system transmit power, via deriving closed-form expression for asymptotic EASR and compact expression for asymptotic SOP. Numerical results have verified the correctness of our theoretical analysis and proved that the FD-BBJ strategy applied in the UAV-aided wireless communication system can help achieve considerable secrecy performance gain.
Radio Resource Management for Cellular-Connected UAV: A Learning Approach
Published on IEEE Transactions on Communications (TCom), Mar 2023.
Integrating unmanned aerial vehicles (UAVs) into existing cellular networks encounters lots of challenges, among which one of the most striking concerns is how to achieve harmonious coexistence of aerial transceivers, inter alia, UAVs, and terrestrial user equipments (UEs). In this paper, a cellular-connected UAV network is focused, where multiple UAVs receive messages from base stations (BSs) in the down-link, while BSs are serving ground UEs in their cells. For effectively managing inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first $p$-tier based RB coordination criterion is proposed and adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design minimizing UAV’s ergodic outage duration (EOD) is investigated. To cope with conventional optimization techniques’ inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is initiated, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in discrete action domain and beamforming design in continuous action regime, respectively. The hybrid D3QN-TD3 solution is trained via interacting with the considered outer and inner environments in an online centralized manner so that it can then help achieve the suboptimal EOD minimization performance during its offline decentralized exploitation phase. Simulation results have illustrated the effectiveness of the proposed hybrid D3QN-TD3 algorithm, compared to several representative baselines.
2022
Covertness-Aware Trajectory Design for UAV: A Multi-Step TD3-PER Solution
Published on IEEE International Conference on Communications (ICC), May 2022.
In the presence of Warden’s detection, a maximization problem on transmission throughput from unmanned aerial vehicle (UAV) to legitimate nodes is considered and solved via UAV trajectory design, subject to covert, velocity and mobility constraints. With the building-distribution-based pathloss model and the Warden’s uncertain location model, the formulated optimization problem is challenging to be tackled through standard offline optimization methods. Alternatively, a twin delayed deep deterministic policy gradient (TD3) approach enhanced by multi-step learning and prioritized experience replay (PER) techniques, termed as multi-step TD3-PER, is proposed to help the UAV adaptively select velocity from continuous action space. Numerical results demonstrate the effectiveness of the proposed multi-step TD3-PER solution and showcase the corresponding superiorities against provided baselines.
Intelligent UAV Navigation: A DRL-QiER Solution
Published on IEEE International Conference on Communications (ICC), May 2022.
The conference version of IEEE TWC journal paper entitled 'Path Planning for Cellular-Connected UAV: A DRL Solution With Quantum-Inspired Experience Replay'.
Path Planning for Cellular-Connected UAV: A DRL Solution With Quantum-Inspired Experience Replay
Published on IEEE Transactions on Wireless Communications (TWC), Apr. 2022.
In cellular-connected unmanned aerial vehicle (UAV) network, a minimization problem on the weighted sum of time cost and expected outage duration is considered. Taking advantage of UAV’s adjustable mobility, a UAV navigation approach is formulated to achieve the aforementioned optimization goal. Conventional offline optimization techniques suffer from inefficiency in accomplishing the formulated UAV navigation task due to the practical consideration of local building distribution and directional antenna radiation pattern. Alternatively, after mapping the navigation task into a Markov decision process (MDP), a deep reinforcement learning (DRL)-aided solution is proposed to help the UAV find the optimal flying direction within each time slot, and thus the designed trajectory towards the destination can be generated. To help the DRL agent commit a better trade-off between sampling priority and diversity, a novel quantum-inspired experience replay (QiER) framework is proposed, via relating experienced transition’s importance to its associated quantum bit (qubit) and applying Grover iteration based amplitude amplification technique. Compared to several representative DRL-related and non-learning baselines, the effectiveness and supremacy of the proposed DRL-QiER solution are demonstrated and validated in numerical results.
2021
Intelligent Trajectory Planning in UAV-Mounted Wireless Networks: A Quantum-Inspired Reinforcement Learning Perspective
Published on IEEE Wireless Communications Letters (WCL), Jun. 2021.
In this letter, we consider a wireless uplink transmission scenario in which an unmanned aerial vehicle (UAV) serves as an aerial base station collecting data from ground users. To optimize the expected sum uplink transmit rate without any prior knowledge of ground users (e.g., locations, channel state information and transmit power), the trajectory planning problem is optimized via the quantum-inspired reinforcement learning (QiRL) approach. Specifically, the QiRL method adopts novel probabilistic action selection policy and new reinforcement strategy, which are inspired by the collapse phenomenon and amplitude amplification in quantum computation theory, respectively. Numerical results demonstrate that the proposed QiRL solution can offer natural balancing between exploration and exploitation via ranking collapse probabilities of possible actions, compared to the traditional reinforcement learning approaches that are highly dependent on tuned exploration parameters.
2020
Harvest-and-Opportunistically-Relay: Analyses on Transmission Outage and Covertness
Published on IEEE Transactions on Wireless Communications (TWC), Aug. 2020.
To enhance transmission performance, privacy level, and energy manipulating efficiency of wireless networks, this article initiates a novel simultaneous wireless information and power transfer (SWIPT) full-duplex (FD) relaying protocol, named harvest-and-opportunistically-relay (HOR). Due to the FD characteristics, the dynamic fluctuation of relay's residual energy is difficult to quantify and track. To solve this problem, the Markov Chain (MC) theory is invoked. Furthermore, to improve the privacy level of the proposed HOR relaying system, covert transmission performance analysis is performed, where closed-form expressions of the optimal detection threshold and minimum detection error probability are derived. Last but not least, with the aid of stationary distribution of the MC, closed-form expression of transmission outage probability is calculated, based on which transmission outage performance is analyzed. Numerical results have validated the correctness of analyses on transmission outage and covertness. The impacts of key system parameters on the performance of transmission outage and covertness are given and discussed. Based on mathematical analysis and numerical results, we showcase that the proposed HOR model can not only reliably enhance the transmission performance via smartly managing residual energy but also efficiently improve the privacy level of the legitimate transmission party via dynamically adjusting the optimal detection threshold.
2018
Artificial Noise Aided Precoding With Imperfect CSI in Full-Duplex Relaying Secure Communications
Published on IEEE ACCESS, July 2018.
In Rayleigh fading channels, to enhance the secrecy performance of wireless communication systems and efficiently disturb the interception of eavesdroppers, the multiple-antenna source node utilizes the artificial noise aided precoding (ANP) strategy with imperfect channel state information to emit the confidential information and the artificial noise simultaneously. Besides, the two-antenna decode-and-forward relay node applies the full-duplex (FD) relaying protocol, and the destination node which contains multiple antennas adopts the maximum ratio combining technique. Taking into account the existence of self-interference at the relay, the closed-form expression of approximate ergodic achievable secrecy rate (EASR) for any value of antenna number and that of exact EASR in the case of large-scale antenna array are derived, respectively. To extract more distinct insights from the considered system and hence obtain some simple and meaningful conclusions, the asymptotic performance analyses in two different asymptotic cases are studied. The numerical simulations validate the correctness of our theoretical derivation and analysis, which indicates that the ANP scheme combined with the FD relaying can achieve considerable secrecy performance.
Antenna Mode Switching for Full-Duplex Destination-Based Jamming Secure Transmission
Published on IEEE ACCESS, Jan. 2018.
We investigate the secrecy rate optimization problem in a wiretap channel with a single-antenna source, a single-antenna eavesdropper, and a multiple-antenna full-duplex (FD) destination. To fully utilize the spatial degrees-of-freedom of multiple antennas, the function of antennas at the destination is not predefined, i.e., each antenna can operate in a transmit or receive mode. We propose a low-complexity near-optimal joint optimization scheme by jointly applying the dynamic antenna mode switching (AMS) and optimal power allocation (OPA) techniques, to maximize the secrecy rate of the FD destination-based jamming (DBJ) system. The proposed joint optimization scheme is valid for two different eavesdropping channel state information (ECSI) availability cases, i.e., instantaneous ECSIs and statistical ECSIs. Specifically, closed-form expressions of OPA factors are first derived, and then the optimal transmit and receive antennas sets at the destination are determined by combining the OPA factor and applying a greedy-search-based AMS approach for both ECSIs availabilities, respectively. Moreover, through complexity analysis, the search complexity of the proposed scheme is proven to be significantly reduced compared with the exhaustive searching method. Simulation results verify the secrecy performance superiority of the proposed scheme over the conventional FD-DBJ method.
2017
Secrecy Performance Analysis of Artificial Noise Aided Precoding in Full-Duplex Relay Systems
Published on IEEE Global Communications Conference (GLOBECOM), Dec. 2017.
In Rayleigh fading channels, we investigate the secrecy performance of a full-duplex relay secure transmission system. To improve the secrecy capacity of the system and efficiently interfere the interception of the eavesdropper, the multiple-antenna source applies the artificial noise aided precoding (ANP) scheme to broadcast the intended signal and the artificial noise simultaneously, and the decode-and-forward relay operates in full- duplex mode. To improve the received signal-to-noise ratio (SNR), the multiple-antenna destination applies maximum ratio combining (MRC) strategy. In the presence of self-interference at the relay, the approximate closed-form expression of ergodic achievable secrecy rate (EASR) for any values of antenna number and the exact closed-form expression of EASR for large-scale antennas array were derived respectively. Both the theoretical analysis and numerical simulations show that the ANP combined with full-duplex scheme can achieve considerable secrecy performance gain.