Is ChatGPT better than Gemini?

Share

Today I tried to use AI chatbots to explain a relatively simple concept to me that I wasn't fully grasping: what prevents a proxy (also known as a "VPN" these days) from intercepting traffic from your web browser to the website, and then pretending to be the website? Since they are in the middle of the connection, they should be able to pretend that they are the website to you, and then pretend that they are you to the website, changing which messages you sent to the website, and reading the messages the website sent back to you. For this I used both ChatGPT and Google's Gemini. I started with Gemini since I was just googling it, but then I thought... "Google keeps showing me these AI summaries all the time, surely it will be able to explain this concept to me in a summarized manner?"

Well, that didn't work the way I'd hoped, but it revealed an important distinction between these two services I'd like to share with you. It's possible that in the future these peculiarities will change. The software reviewed was Gemini 3 (according to Gemini), and ChatGPT

Gemini

When I asked Gemini to explain me how VPNs worked, it kept generating very similar bullet point lists with headings. It kept repeating the same information, and there was no way to make it narrow down to explain the steps I was actually confused about.

After a few questions, I understood that the proxy can't pretend to be the destination server because the first packet it receives is already encrypted via TLS. But that is a bit weird. For it to be encrypted, it needs the destination server's cryptographic key, and if it got that, it would have to be through the proxy which is in the middle of the connection. In other words, theoretically the proxy could simply take the server's key for themselves, and send to you the proxy's own key. Then all it would need to do is decrypt the packets it received from you with their own key, and re-send them to the destination with the destination key.

Of course I assume there was something that prevents that from happening, I was just not sure what would that be, and Gemini wasn't telling me.

Even as I asked to describe the steps between connecting to the proxy and sending the first encrypted TLS packet, it still kept giving me a lot of information that was only tangentially related. There was no way to narrow the context to just the small part I wasn't understanding. It kept repeating the part that it already had explained.

It seemed designed to not elaborate further, and just give vague explanations of the concept.

This is the input and the headings of the output. I won't write the whole output in this article, and you really shouldn't assume it's correct output either.

explain the steps between the web browser first connecting to a website via proxy and it encrypting the first packet.
  1. Establish the Proxy Control Link
  2. Request the Tunnel (CONNECT or SOCKS Request)
  3. Initiate the TLS Handshake (The First "Packet")
  4. Identity Verification and Key Exchange
  5. First Fully Encrypted Application Data
Gemini

ChatGPT

When I asked the ChatGPT the question that Gemini refused to elaborated, it created a long list of steps as headings explaining the entire process in detail.

explain the steps between a web browser connecting to a website through a socks5 proxy and it sending the first TLS encrypted packet to the proxy
  1. Browser resolves how to reach the proxy
  2. TCP connection to the SOCKS5 proxy
  3. SOCKS5 protocol negotiation (plaintext)
  4. SOCKS5 CONNECT request (plaintext)
  5. Proxy resolves DNS (usually)
  6. Proxy establishes TCP connection to the website
  7. Tunnel is established (raw TCP forwarding)
  8. TLS handshake begins (first encrypted data comes later)
  9. ServerHello and key exchange (partially plaintext)
  10. TLS key agreement completes
  11. First TLS-encrypted application packet is sent
ChatGPT

Admittedly, the input is a bit different, but that's not what is important.

By the way, even after asking these questions, I still hadn't figured out what was preventing the man-in-the-middle attack from happening, but after asking several other questions to ChatGPT. it finally clicked: the browser doesn't accept any cryptographic key that it receives, it needs to be a key that matches the domain name it's trying to access. But what prevents the proxy from lying about which domain name the key belongs to? The browser only trusts keys that are signed by a Certificate Authority (CA). And the CA's only sign a key after confirming it belongs to a domain name. In other words, there is a trust system outside of the cryptographic system that is responsible for making sure that a key actually belongs to the domain name it says it comes from. In order for the proxy to forge the server's key, it would need to get a CA to sign their fake key, or, alternatively, it would need to actually steal the private key of the server.

Differences between Gemini and ChatGPT

Does this mean that ChatGPT is better than Gemini? Unfortunately, things aren't so simple.

Although Gemini's answers weren't very useful, it seems the main reason for that is that it's first and foremost a search engine, or rather, a search-summarizer. Gemini always searches the web first before giving an answer. Even asking it to generate a poem will trigger a web search. And its answers are often full of cited articles.

It's very important to note that just because Gemini cites a webpage, that doesn't actually mean the website contains the cited assertion. Due to how neural networks work, the output of LLM's is always imprecise and unreliable. To put it in an anthropomorphizing way, Gemini "misinterprets" what the article says, and gives us its misinterpretation while citing an article that may, sometimes, even say the completely opposite of what it just told us. For any serious work, you should simply ignore the LLM output and check the cited articles. The only use of the LLM output is then to tell you what the linked page may be about.

On the other hand, ChatGPT just generated the whole answer out of thin air and didn't even cite anything. Given the imprecise and unreliable nature of LLM's, that means the entire answer may be just random gibberish full of completely wrong information, and there is no convenient way to check. We'd have to search for every assertion that we care about individually to verify the claims, and hope to find an article written by a real human being in 2026 instead of just random AI-generated articles.

Consequently, the two choices presented in this article are:

  1. Gemini: an AI chatbot that is vague, cites sources, but the sources may not actually contain the informtion it's citing.
  2. ChatGPT: an AI chatbot that is precise, but may generate lots of information without any sources, and it can also be completely wrong.

Personally, I actually prefer ChatGPT. I think Gemini is pointless, because I'd rather just have the good old search results, which cite the webpages verbatim, than something that attempts to "summarize" information but ends up transforming and corrupting it.

Then why prefer ChatGPT? Because both are unreliable. It wouldn't matter even if ChatGPT were wrong 99% of the time and Gemini were wrong only 1% of the time. So long as it's not correct 100% of the time, you are ALWAYS going to have to double check the information that you care about. There is no such thing as "most probably correct information." Considering this, I prefer the program that is better at generating answers. It doesn't matter if those answers are are more wrong or less wrong than the other program, because I'm going to have to double check everything either way to make sure.

Written by Noel Santos.

About the Author

I'm a self-taught Brazilian programmer graduated in IT from a FATEC. In a world of increasingly complex and essential computers, I decided to use my technical expertise in hardware, desktop applications, and web technologies to create an informative resource to make PC's easier to understand.

View Comments