What is the «scraping» function on your newly purchased RG351 model? Learn the what, why, and how of scraping on your new retro gaming handheld with this handy short guide.
What IS game (metadata) scraping?
A feature present in all of the RG351 models at base – this being the RG351P, RG351M, RG351V and RG351MP – is the ability to do perform an action known as «scraping».
Game files, whether they be ROMs, BINs, ISOs, or otherwise – do not come with any box art or extra information attached. They are purely just the games and all of the content inside of them.
When copy/pasted directly to the device, provided you have put it in the correct folder, the game will appear «as-is» with its given filename. You could upload a Sonic The Hedgehog ROM and call it «Grand_Theft_Auto_27», leaving yourself perpetually puzzled every time you open the game on your £100 device and are not greeted with an AAA Blockbuster title from 80 years in the future.
To get around this, most operating systems make use of something known as «metadata«. Metadata is data used to describe what another piece of data is. For games, this would typically include information such as the actual title (which for a variety of reasons, isn’t always the best thing to use as a filename), release date, and even visual media like box art.
Not all of us are collectors however, and we’re all also busy people. So it’s unlikely we’ll have time to manually enter all this metadata for our expansive game collections. This is where game preservation societies and scraping come in.
Scraping is an automated process that enables you to gather metadata for hundreds upon thousands of games with very little effort.
How does game scraping work?
By default, all of the metadata for a given system is stored in a file called «gamelist.xml». You can find one for each folder in the system.
This file contains rows upon rows of data that describe various aspects of the game file they point to (denoted by the <path> tag), and each game can be adjusted to feature as little or as much information as you’d like.
How the system works is that, when the device boots, it will look at these .xml files and use them to populate the lists of games displayed by the system. It will then automatically fill in the user interface as defined by the theme, with the metadata from the defined tags.
How this metadata is shown is entirely dependent on your theme and the options you have selected for it (which can also be adjusted freely).
Scraping is the process of automatically filling out these metadata files with hundreds upon thousands of entries, usually based on the filename of the title.
How do I scrape for game (meta)data?
Currently, the largest online database publicly available is ScreenScraper. On this site, there are hundreds upon thousands of games that have had information, box art, and even small preview gameplay videos for them be manually uploaded and curated by hundreds of generous volunteers.
There are also alternative sites such as TheGamesDB who perform similar services.
By default, the RG351s will query ScreenScraper for metadata downloads.
ScreenScraper promptly will automatically disable scraping from unregistered clients when the server load is too high, as when too many requests are made, the server could potentially crash.
If at the time of publishing, scraping is currently disabled for unregistered clients, you can circumvent this by signing up on the ScreenScraper website, and then inputting your credentials into the RG351’s scraping interface. There is a hard cap on the amount of data you can scrape per day, but this can be raised for people who donate to the site.
To access the scraping function, press the start button while in the main menu, and select the «Scrape» option.
Before you scrape for metadata, it is possible to define exactly what you wish to be downloaded.
Defining Metadata
Some notable tags/fields you can define are as follows:
- Image Source
- This is the main image that will be used to represent the game in most themes.
- You can choose from the following images to use:
- Screenshot – A random screenshot of the game.
- Title Screenshot – A screenshot of the games’ title screen.
- Box 2D – A flat scan of the game’s box art
- Box 3D – The game’s box art viewed from an angled 3D perspective
- Fan Art – Self-explanatory
- Mix V1 – A combination of Box 3D and Screenshot, alongside a logo.
- Mix V2 – A combination of Box 3D and Screenshot, alongside a logo and mock physical media (CD, cartridge, etc.)
- Box Source
- This is purely a field for the box-art in case you are using one of the Mixed image sources.
- Logo Source
- Similar to Box Source, but for the logo itself.
- Opting for «Wheel» will download a stand-alone logo of the game.
- Opting for «Marquee» will download the logo in a standardized format, making things more visually consistent.
- Similar to Box Source, but for the logo itself.
- Ratings
- An arbitrary rating for the game as defined by the ScreenScraper community. Useful for helping decide what to play at a glance.
- Videos
- You can download a short gameplay video that will play in most themes after a short delay. Acting as a kind of preview.
- For large collections, this option can really eat into the available storage space, so exercise caution.
There are also other options to download fanart, game manuals, and more. But depending on your firmware and theme, you may not even be able to view these. So they are often a waste of space.
Missing Medias?
Before going ahead with the scraping, you will be given one final option to select what specific platforms you wish to scrape for, and if you want to only scrape for «missing medias».
If you opt to scrape for missing medias only, any game that already has an entry for a tag will not be scraped. This will save you a lot of time if you only adding a few titles to a game collection that already has a lot of metadata.
Once you’ve set these options, all you need to do is hit the «Scrape Now» button. The scraping process will carry out automatically in the background. Depending on the number of games and types of metadata you are scraping, this process could either take a few seconds, or multiple hours.
We would advise not playing the device during this period.
Can I scrape for game metadata on PC?
You can! While there are many tools out there to do this, one of the most popular is Skraper. With these tools you can prepare your SD card before putting it into the system. This saves you time, and increases the granularity of the customization available.
How to use Skraper in particular is beyond the scope of this article. But you can check out a handy video guide from popular youtuber Retro Game Corps.
Conclusion
With scraping, it is possible to really «pretty up» your collection of games. Making them nicer to look at and easier to browse for both yourself and anyone who may be using the device.
We highly recommend making use of scraping in order to really enhance your experience. But do go easy on the ScreenScraper servers.
If you’re interested in purchasing an RG351 device of your own, we encourage you to check out our range of ANBERNIC handhelds at droix.co.uk. Or perhaps check out some of our best retro gaming handhelds here on the DroiX blog.
