Recent data-driven approaches using neural networks have shown promising results for pedestrian trajectory prediction. These algorithms outperform the knowledge-based and physics-based models in terms of distance error. However, it has been observed that the neural networks produce too many collisions and pedestrian overlaps, leading to unrealistic predictions. To address this problem, we propose in this contribution a hybrid extension of the Social-LSTM data-driven approach by introducing a collision loss in the training. The collision loss is provided by an interaction energy based on the timeto-collision with the neighbors. The predictions are evaluated and compared to the Social-LSTM model using both distanceerror metrics and collision metrics. The results show that collision and pedestrian overlap in the predicted trajectories decreases exponentially as the collision loss weight in the training increases, while the displacement error remains approximately constant. These preliminary results make the proposed hybrid algorithm a promising approach for realistic pedestrian prediction, especially in high-density situations.