The question if God exists or not is usually not what the atheist debate is about. In the end, everybody believes what they want. The real question is: Is the belief in God a rational point of view?
To me, it clearly is.
The documented defense of the rationality of the faith started with Aristotle and eventually culminated in the Kalam Cosmological Argument (KCA). The main advantage of the KCA is that it is essentially a mathematical theorem:
Time is finite since it progresses by addition, while forming an actual infinite through successive addition is impossible. This establishes the existence of a beginning of times. Under the assumption of generalized causality, there is always a cause for an event, which must strictly precede the event in time. Assuming that supertasks cannot exist, and if we follow back the causal chain of events, necessarily in a finite number of steps, we will arrive in the first event. The cause for this first event is the first cause of the entire universe. Because of the law of the indiscernibles, there can only be one such first cause in the initial singularity from which timespace started expanding.
If you are interested in the deeper details, William Lane Craig is quite a famous contemporary expert on the KCA.
The first cause theorem is the central claim of the Abrahamic religions, that is, Judaism, Christianity, and Islam. In Genesis 1:1, you can find one of the canonical examples in which the scriptures claim the first cause theorem: “In the beginning God created the heavens and the earth.”