ROM - Revision history

Wtv-411: Add debug symbols info

2023-02-14T01:21:59Z

Add debug symbols info

← Older revision		Revision as of 21:21, 13 February 2023
Line 89:		Line 89:
	};		};
	</pre-->		</pre-->

			=== Debug Symbols ===
			WebTV/MSN TV firmware can come with proprietary symbol files meant to aid in debugging. These files have a .sym extension and they mostly come included in the ROMFS of debug builds as <code>ROM/webtv.sym</code>, although there are symbol files known to exist for production builds, albeit only as individual files. While the symbol files are proprietary, a tool exists originally written by WebTV hacker eMac that can import these symbols into IDA Pro via a script. A fork of the script by Sgeo also exists that converts the symbol files into a Ghidra-compatible format. Both are available on GitHub as of writing ([https://github.com/Sgeo/webtv-symbol-importer/tree/59eb4ba10503035e5fbbf2770aebbc5cbdeea425 IDA], [https://github.com/Sgeo/webtv-symbol-importer Ghidra]).

	=== Upgrades ===		=== Upgrades ===

Wtv-411 at 13:43, 13 February 2023

2023-02-13T13:43:22Z

← Older revision		Revision as of 09:43, 13 February 2023
Line 1:		Line 1:
	ROM in the WebTV/MSN TV world is used to refer to two things. It can refer to the type of chips first-gen boxes use to store and execute firmware, whether it be boot ROMs stored on read-only ROM chips that initialize the box hardware when executed on boot up, or the main client software, also known as app ROMs, that's stored on ~~non-volatile~~ flash ROM. It can also refer to the firmware itself that boxes execute.		ROM in the WebTV/MSN TV world is used to refer to two things. It can refer to the type of chips first-gen boxes use to store and execute firmware, whether it be boot ROMs stored on read-only ROM chips that initialize the box hardware when executed on boot up, or the main client software, also known as app ROMs, that's stored on a flash ROM chip (or "NVRAM"), or on a hard drive in WebTV Plus boxes. It can also refer to the firmware itself that boxes execute.

	== Firmware ==		== Firmware ==
	Firmware, in terms of WebTV/MSN TV, is the software that allow boxes to start up and run the main WebTV/MSN TV browser, allowing them to connect to the WebTV/MSN TV service. For our purposes, we'll use the term "firmware" to collectively refer to both boot ROMs and app ROMs.		Firmware, in terms of WebTV/MSN TV, is the software that allow boxes to start up and run the main WebTV/MSN TV browser, allowing them to connect to the WebTV/MSN TV service. For our purposes, we'll use the term "firmware" to collectively refer to both boot ROMs and app ROMs.

	Firmware for first-gen WebTV/MSN TV devices are built as MIPS binaries, possibly designed for the III/IV revision of the architecture. While the size of the client binaries can vary, with the largest one we've seen being an 18.5 MB UltimateTV ROM, boot ROMs, regardless of the hardware they were designed for, always seem to have a consistent 2 MB ~~size~~.		Firmware for first-gen WebTV/MSN TV devices are built as MIPS binaries, possibly designed for the III/IV revision of the architecture. While the size of the client binaries can vary, with the largest one we've seen being an 18.5 MB UltimateTV ROM, boot ROMs, regardless of the hardware they were designed for, always seem to have a consistent size of 2 MB, or 4 MB for Japanese units.

	For client software, since those are designed to be stored on non-volatile memory, the storage medium used on the boxes (NVRAM, hard disk) can be overwritten to use a different ROM. This is usually done for upgrades.		For client software, since those are designed to be stored on non-volatile memory, the storage medium used on the boxes (NVRAM, hard disk) can be overwritten to use a different ROM. This is usually done for upgrades.

Wtv-411: Add info on boot ROM

2023-02-04T21:20:17Z

Add info on boot ROM

← Older revision		Revision as of 17:20, 4 February 2023
Line 2:		Line 2:

	== Firmware ==		== Firmware ==
	Firmware, in terms of WebTV/MSN TV, is the software that allow boxes to start up and run the main WebTV/MSN TV ~~client software~~, ~~which include the browser and capabilities~~ to connect to the ~~proprietary~~ WebTV/MSN TV service. For our purposes, we'll use the term "firmware" to collectively refer to both boot ROMs and app ROMs.		Firmware, in terms of WebTV/MSN TV, is the software that allow boxes to start up and run the main WebTV/MSN TV browser, allowing them to connect to the WebTV/MSN TV service. For our purposes, we'll use the term "firmware" to collectively refer to both boot ROMs and app ROMs.

	Firmware for first-gen WebTV/MSN TV devices are built as MIPS binaries, possibly designed for the III/IV revision of the architecture. While the size of the client binaries can vary, with the largest one we've seen being an 18.5 MB UltimateTV ROM, boot ROMs, regardless of the hardware they were designed for, always seem to have a consistent 2 MB size.		Firmware for first-gen WebTV/MSN TV devices are built as MIPS binaries, possibly designed for the III/IV revision of the architecture. While the size of the client binaries can vary, with the largest one we've seen being an 18.5 MB UltimateTV ROM, boot ROMs, regardless of the hardware they were designed for, always seem to have a consistent 2 MB size.

	For client software, since those are designed to be stored on non-volatile memory, the chips ~~can~~ be ~~programmed~~ to store ~~different client software~~, ~~usually~~ for upgrades.		For client software, since those are designed to be stored on non-volatile memory, the storage medium used on the boxes (NVRAM, hard disk) can be overwritten to use a different ROM. This is usually done for upgrades.

			=== Boot ROM ===
			On real hardware, the boot ROM is what's first executed as the box starts up. On all box models except for the original 1996 boxes (bf0app), the boot ROM is loaded at address 0x9fc00000 directly from ROM chips on the motherboard. Based on the header in bf0app boot ROMs, the original Classic models likely loaded the boot ROM at address 0x3c037fff, although this has yet to be proven to be the case on the hardware. The US boxes use 2 ROM chips on U0503 and U0504 to store the boot ROM, while Japanese boxes use all 4 ROM chips installed from U0501 - U0504. The contents of the boot ROM are "16-bit stripped", meaning that each pair of the ROM chips store alternating 16-bit chunks of the ROM data (ROM chip 1 stores the first 16 bits of data, ROM chip 2 stores the second 16 bits, ROM chip 1 stores the third 16 bits, rinse and repeat). Japanese boxes use the extra 2 ROM chips to accommodate for the larger size of its boot ROM, using the same 16-bit stripping method to store data.

			Once the boot ROM executes, it will initialize the CPU and RAM. On all box models after the Classic, the boot ROM will then unpack a "mini browser" program and load it into RAM. The minibrowser is a stripped down version of the WebTV browser, likely based on an unreleased 2.0 version of the firmware, and is what will verify and load the main firmware into memory. On any WebTV/MSN TV box with a hard drive (this probably applies to boxes with a DiskOnChip as well), the main firmware is stored at offset 0x880600 on the hard drive, with another firmware being stored at 0x80600 as a backup. If the firmware can be located on the box's storage medium and its checksums are valid, then it will be loaded into RAM and execution is passed over to the firmware. If attempting to load the firmware fails for any reason, then the boot ROM will enter a recovery mode, where it attempts to connect to the WebTV/MSN TV service to download a firmware onto the box. On Classic boxes, this is a custom program with its own interface that talks to the service to flash the latest firmware. On every other box model, this uses the minibrowser to handle client upgrades.

	=== ROM Header ===		=== ROM Header ===

Wtv-411: /* ROM Header */

2023-01-31T01:20:32Z

ROM Header

Wtv-411: /* ROM Header */

2023-01-29T17:23:41Z

ROM Header

← Older revision		Revision as of 13:23, 29 January 2023
Line 20:		Line 20:
	\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.		\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.
	\|-		\|-
	\| ROM length \|\| uint32 \|\| 0x0c \|\| Length of ROM including ROMFS data. Not exactly sure how this is calculated		\| ROM length \|\| uint32 \|\| 0x0c \|\| Length of ROM starting from offset 0x0C, including ROMFS data. Not exactly sure how this is calculated
	\|-		\|-
	\| Code length \|\| uint32 \|\| 0x10 \|\| Length of ROM code only		\| Code length \|\| uint32 \|\| 0x10 \|\| Length of ROM code only, starting from offset 0x0C
	\|-		\|-
	\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number		\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number

Wtv-411: /* ROM Header */

2023-01-24T16:24:19Z

ROM Header

← Older revision		Revision as of 12:24, 24 January 2023
Line 20:		Line 20:
	\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.		\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.
	\|-		\|-
	\| ROM length \|\| uint32 \|\| 0x0c \|\| Length of ROM data~~, excluding ROMFS~~. Not exactly sure how this is calculated		\| ROM length \|\| uint32 \|\| 0x0c \|\| Length of ROM including ROMFS data. Not exactly sure how this is calculated
	\|-		\|-
	\| Code length \|\| uint32 \|\| 0x10 \|\| Length of ROM code?		\| Code length \|\| uint32 \|\| 0x10 \|\| Length of ROM code only
	\|-		\|-
	\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number		\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number

imported>Admin: /* ROM Header */

2022-09-11T14:35:12Z

ROM Header

New page

ROM in the WebTV/MSN TV world is used to refer to two things. It can refer to the type of chips first-gen boxes use to store and execute firmware, whether it be boot ROMs stored on read-only ROM chips that initialize the box hardware when executed on boot up, or the main client software, also known as app ROMs, that's stored on non-volatile flash ROM. It can also refer to the firmware itself that boxes execute.

== Firmware ==
Firmware, in terms of WebTV/MSN TV, is the software that allow boxes to start up and run the main WebTV/MSN TV client software, which include the browser and capabilities to connect to the proprietary WebTV/MSN TV service. For our purposes, we'll use the term "firmware" to collectively refer to both boot ROMs and app ROMs.

Firmware for first-gen WebTV/MSN TV devices are built as MIPS binaries, possibly designed for the III/IV revision of the architecture. While the size of the client binaries can vary, with the largest one we've seen being an 18.5 MB UltimateTV ROM, boot ROMs, regardless of the hardware they were designed for, always seem to have a consistent 2 MB size.

For client software, since those are designed to be stored on non-volatile memory, the chips can be programmed to store different client software, usually for upgrades.

=== ROM Header ===
All firmware ROMs appear to have a header at the very start of the ROM that contain jump instructions pointing to the main code and information about the ROM, such as a checksum, load address, and build number. This header is 0x44 bytes long, and all integer values are packed in big-endian order:

{| class="wikitable"
! Field Name !! Type !! Offset !! Description
|-
| Jump instruction || uint32 || 0x00 || MIPS jump instruction to build entry point. If this field has the value <code>0x10000000</code>, the build has compressed code and data (usually at offset 0x200) that has to be decompressed with an algorithm specified later in the ROM header
|-
| "Pre-jump instruction" || uint32 || 0x04 || Another MIPS instruction(?). Usually this is just a NOP instruction
|-
| Code checksum || uint32 || 0x08 || Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.
|-
| ROM length || uint32 || 0x0c || Length of ROM data, excluding ROMFS. Not exactly sure how this is calculated
|-
| Code length || uint32 || 0x10 || Length of ROM code?
|-
| ROM build number || uint32 || 0x14 || ROM build number
|-
| Heap data address || uint32 || 0x18 || Address of initial heap data in RAM
|-
| Heap data size || uint32 || 0x1c || Uncompressed size of initial heap data
|-
| "Heap free size" || uint32 || 0x20 || ???
|-
| ROMFS base address || uint32 || 0x24 || Base address for ROMFS data relative to the address specified in the "ROM base address" field. Don't know how this is used. In UltimateTV client ROMs, this address is always set to 0x4e6f4653 (<code>NoFS</code> in ASCII)
|-
| "unknown1" || uint32 || 0x28 || In most builds, this value is either a string specifying a compression algorithm if the build is compressed, or nulled. Some ROMs, however (i.e., <code>bf0app</code> boot ROMs), use this field to store random data and don't have a clear purpose for doing so. Known algorithm strings used for <code>unknown1</code> are <code>lzj0</code>, <code>lzj1</code>, and <code>lzj2</code>
|-
| "unknown2" || uint32 || 0x2c || ???
|-
| ROM base address || uint32 || 0x30 || A base address where the firmware is loaded into in RAM.
|-
| Build flags || uint32 || 0x34 || Bitmask flag value that specifies what type of WebTV/MSN TV ROM was built. See [[#Build Flags|Build Flags]]
|-
| Heap data compressed size || uint32 || 0x38 || Compressed size of initial heap data
|-
| Compressed code address || uint32 || 0x3c || If the ROM code is compressed, this specifies the address of the code relative to the address specified in "ROM base address" field
|-
| Compressed code size || uint32 || 0x40 || Compressed size of ROM code
|}

==== Build Flags ====
Bitmask flags used for the build flags field in the ROM header.

* <code>0x01</code>: WebTV/MSN TV build can handle compressed heap data
* <code>0x04</code>: WebTV/MSN TV build is a debug build
* <code>0x10</code>: Assumed to mark a WebTV/MSN TV build as a Windows CE build. Appears in build flags for UltimateTV app ROMs
* <code>0x20</code>: Appears to mark a WebTV/MSN TV build as a satellite build. Appears in build flags for both DishPlayer boot ROMs and UltimateTV app ROMs



=== Upgrades ===
WebTV/MSN TV boxes have the ability to download upgrades to the client software from the WebTV/MSN TV network and install them onto themselves. These are known as flash upgrades, namely due to the fact that they're installed onto the flash ROM that boxes boot into. Even outside the realm of [[WTVP]], we have next to no information on how the boxes handled actually installing these upgrades other than simply flashing them onto flash ROM. What we do know so far is that supposedly, these flash upgrades were organized by class of WebTV/MSN TV box, split into pieces, compressed, and digitally signed before being published onto the WebTV/MSN TV network [https://fadden.com/resume.html]. WNI also had other fancy ways of classifying flash upgrades that for the time being won't be elaborated on here. We advise you to read this article by Andy McFadden instead to get a glimpse into how this was handled: https://fadden.com/tech/software-update.html.

==== Availability ====
Before 2021, very little had been done to fully archive the client upgrade ROMs publicly. We are aware that someone operating a private third-party WebTV/MSN TV server had most of these available for download directly through the WebTV/MSN TV protocol supported by the server itself, so it's not like no one tried. Sadly, due to the server being completely private in nature and the difficulty of being able to reach out to the people involved with operating the server to get any missing upgrade ROMs, some of these ROM sets may never see the light of day, at least for a while. This might put them at the risk of being completely lost, and we already have enough to deal with right now concerning the preservation of WebTV/MSN TV information.

With the blessings of eMac and Zefie, however, we have been able to successfully preserve a great amount of builds of the WebTV/MSN TV client software and boot ROMs, which while supposedly only cover the stock (a.k.a. "artemis") versions of the ROMs, span several hardware types and are also available in both their upgrade (chunked) and complete forms. You can currently find these ROMs on [https://archive.org/details/WebtvMsntvRomCollection Archive.org]. This isn't the end of the story, though. There are obviously more builds that we have yet to see surface, and we've yet to see any samples of any vendor-specific (Sony, Philips, etc.) ROMs in any form. This is why we're actively seeking anyone who might have this data among other things so that we can not only obtain it, but document and publish it for future generations. If you have any '''complete''' ROM sets or anything else important on hand, don't hesitate to release them or [[Wanted|send it over to us]].

=== Hardware Classes ===
Hardware classes are identifiers that define the hardware components or general type a firmware ROM is compatible with. Aside from hardware classes <code>bf0app</code> and <code>bfeapp</code>, hardware classes usually specify the region, base type, storage medium, and memory size in megabytes it targets, with some specifying softmodem support and a particular CPU they target. Not everything about the hardware class has been figured out right now, but we have a good sense of how it works. The typical format of a hardware class string looks like this:

{REGION}-{TYPE}-{MEDIUM}-0MB-{MAX-MEGS}MB-[softmodem-CPU''NNNN'']

For a list of currently known hardware classes: [[List of WebTV (MSN TV) Hardware Classes]]

=== Vendors ===
The WebTV/MSN TV service offered firmware ROMs in both stock (Artemis) and vendor-specific forms, the latter being designed for usage with boxes from authorized manufacturers. We aren't aware of any differences between stock ROMs and ROMs made for specific hardware vendors, but it's possible that the vendor-specific ROMs offered by the service were just aliases for the stock ROMs. If you happen to have any information on either, check out the Wanted page on the sidebar to see how you can help contribute.

In the meanwhile, we have a (very barren) list of vendor IDs we have been able to obtain and record here: [[List of WebTV (MSN TV) ROM Vendor IDs]].

← Older revision		Revision as of 21:20, 30 January 2023
Line 14:		Line 14:
	! Field Name !! Type !! Offset !! Description		! Field Name !! Type !! Offset !! Description
	\|-		\|-
	\| Jump instruction \|\| uint32 \|\| 0x00 \|\| MIPS jump instruction to build entry point. If this field has the value <code>0x10000000</code>, the build has compressed code and data (usually at offset 0x200) that has to be decompressed with an algorithm specified later in the ROM header		\| Jump instruction \|\| uint32 \|\| 0x00 \|\| MIPS jump instruction to build entry point. If this field has the value <code>0x10000000</code>, the build has compressed code and data (usually at offset 0x200) that has to be decompressed with an algorithm specified later in the ROM header.
	\|-		\|-
	\| "Pre-jump instruction" \|\| uint32 \|\| 0x04 \|\| Another MIPS instruction(?). Usually this is just a NOP instruction		\| "Pre-jump instruction" \|\| uint32 \|\| 0x04 \|\| Another MIPS instruction(?). Usually this is just a NOP instruction
Line 20:		Line 20:
	\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.		\| Code checksum \|\| uint32 \|\| 0x08 \|\| Checksum of ROM code. eMac had a WebTV/MSN TV Build Info Viewer program on GitHub that contained code to generate these checksums, but the source code is no longer available due to his GitHub account being purged.
	\|-		\|-
	\| ROM length \|\| uint32 \|\| 0x0c \|\| ~~Length~~ of ROM ~~starting~~ from offset 0x0C~~, including~~ ROMFS data. ~~Not exactly sure how~~ this ~~is calculated~~		\| ROM length \|\| uint32 \|\| 0x0c \|\| Mangled length of ROM, including ROMFS data. The length counts data from offset 0x0C and includes ROMFS data. Getting the actual length involves using this algorithm: <code>(ROM_LENGTH - 3) * 4</code>.
	\|-		\|-
	\| Code length \|\| uint32 \|\| 0x10 \|\| ~~Length~~ of ROM code only~~, starting~~ from offset 0x0C		\| Code length \|\| uint32 \|\| 0x10 \|\| Mangled length of ROM code only (all data from offset 0x0C to start of "joeb" padding). Actual length calculated using the same algorithm as ROM length.
	\|-		\|-
	\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number		\| ROM build number \|\| uint32 \|\| 0x14 \|\| ROM build number
	\|-		\|-
	\| Heap data address \|\| uint32 \|\| 0x18 \|\| Address of initial heap data in RAM		\| Heap data address \|\| uint32 \|\| 0x18 \|\| Address of initial heap data in RAM relative to the address specified in "ROM base address" field.
	\|-		\|-
	\| Heap data size \|\| uint32 \|\| 0x1c \|\| Uncompressed size of initial heap data		\| Heap data size \|\| uint32 \|\| 0x1c \|\| Uncompressed size of initial heap data. Likely stores its value in a mangled format like ROM/code length?
	\|-		\|-
	\| "Heap free size" \|\| uint32 \|\| 0x20 \|\| ???		\| "Heap free size" \|\| uint32 \|\| 0x20 \|\| ???
	\|-		\|-
	\| ROMFS base address \|\| uint32 \|\| 0x24 \|\| ~~Base address for~~ ROMFS ~~data~~ relative to the address specified in the "ROM base address" field~~. Don't know how this is used~~. In UltimateTV client ROMs, this address is always set to 0x4e6f4653 (<code>NoFS</code> in ASCII)		\| ROMFS base address \|\| uint32 \|\| 0x24 \|\| Address pointer to start of ROMFS header relative to the address specified in the "ROM base address" field. In UltimateTV client ROMs, this address is always set to 0x4e6f4653 (<code>NoFS</code> in ASCII), as UltimateTV uses the [[CompressFS]] format to store files.
	\|-		\|-
	\| "unknown1" \|\| uint32 \|\| 0x28 \|\| In most builds, this value is either a string specifying a compression algorithm if the build is compressed, or nulled. Some ROMs, however (i.e., <code>bf0app</code> boot ROMs), use this field to store random data and don't have a clear purpose for doing so. Known algorithm strings used for <code>unknown1</code> are <code>lzj0</code>, <code>lzj1</code>, and <code>lzj2</code>		\| "unknown1" \|\| uint32 \|\| 0x28 \|\| In most builds, this value is either a string specifying a compression algorithm if the build is compressed, or nulled. Some ROMs, however (i.e., <code>bf0app</code> boot ROMs), use this field to store random data and don't have a clear purpose for doing so. Known algorithm strings used for <code>unknown1</code> are <code>lzj0</code>, <code>lzj1</code>, and <code>lzj2</code>
Line 42:		Line 42:
	\| Build flags \|\| uint32 \|\| 0x34 \|\| Bitmask flag value that specifies what type of WebTV/MSN TV ROM was built. See [[#Build Flags\|Build Flags]]		\| Build flags \|\| uint32 \|\| 0x34 \|\| Bitmask flag value that specifies what type of WebTV/MSN TV ROM was built. See [[#Build Flags\|Build Flags]]
	\|-		\|-
	\| Heap data compressed size \|\| uint32 \|\| 0x38 \|\| Compressed size of initial heap data		\| Heap data compressed size \|\| uint32 \|\| 0x38 \|\| Compressed size of initial heap data. Likely stores its value in a mangled format as well?
	\|-		\|-
	\| Compressed code address \|\| uint32 \|\| 0x3c \|\| If the ROM code is compressed, this specifies the address of the code relative to the address specified in "ROM base address" field		\| Compressed code address \|\| uint32 \|\| 0x3c \|\| If the ROM code is compressed, this specifies the address of the code relative to the address specified in "ROM base address" field. Boot ROMs have been observed setting this field too, but without appropriately setting the jump instruction. Not sure what the boot ROM does in that case.
	\|-		\|-
	\| Compressed code size \|\| uint32 \|\| 0x40 \|\| Compressed size of ROM code		\| Compressed code size \|\| uint32 \|\| 0x40 \|\| Compressed size of ROM code. Likely stores its value in a mangled format similar to the ROM/normal code length.
	\|}		\|}