So it seems this distribution doesn't seem to have either Chromium or Chrome easily available so I solved this with the following snippet in my buildspec.yml
file.
install: commands: - wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm - sudo yum install -y ./google-chrome-stable_current_x86_64.rpm - sudo ln -s /usr/bin/google-chrome-stable /usr/bin/chromium
Basically it downloads Chrome from Google and then symlinks the chromium to it. After this I can run both Pupeteer and Cypress tests from it.