Sequence Q-Learning Algorithm for Optimal Mobility-Aware User Association

User association in heterogeneous wireless networks is a critical yet challenging problem, especially under user mobility. Conventional optimization-based approaches rely on static or slowly-varying network states and fail to adapt to rapid mobility dynamics. In this paper, we propose a Sequence Q-Learning (SQL) algorithm that formulates mobility-aware user association as a sequential decision-making problem and learns optimal association policies through reinforcement learning. The SQL agent observes network state sequences to capture temporal mobility patterns and selects association actions to maximize long-term quality-of-service (QoS). We analyze the convergence properties of the proposed algorithm and validate its effectiveness through extensive simulations across diverse mobility scenarios. Results show that SQL significantly outperforms conventional greedy and traditional Q-learning baselines in both throughput and handover frequency.