You’ve answered your question in the first line. If you want to test your React app end-to-end, connected to APIs and deployed somewhere, you’d use Cypress.
react-testing-library‘s aimed at a lower level of your app, making sure your components work as expected. With Cypress, your app might be deployed on an environment behind CDNs, using caching, and its data could come from an API. In Cypress you’d also write an end-to-end journey, a happy path through your app that might give you extra confidence once you’ve deployed.