I wrote that thread over on the RetroArch forum, I guess I'll try to explain in case any of it over there isn't clear.

The Wii (and GameCube)'s pixel aspect ratio is 10:11. That is to say, each individual pixel is a rectangle with dimensions of 10*11 (1/11th slimmer than square). The video encoder can scale up to 720 pixels wide, and can be used to correct for the non-square pixels in cases where we want them to be square (like Game Boy Advance emulators).

Assuming (!) that the framebuffer width mGBA uses is 640 wide, correcting for this is as simple as:

11/10*640
=704

... So, set video encoder width to 704. This results in pixels that are 1:1 square, and the video encoder's scaling is really nice; don't mind the non-integer scaling width, the result is good, promise.

Minor point: scaling to 704 is quite wide for the Wii/GCN, it does risk going into the overscan area on some (many) TV sets. This shouldn't be a problem in "Pixel-Accurate" screen mode, since the emulated GBA has plenty of empty space around it for overscan to eat. In "Stretched" mode, though, some people may start losing picture. I'm not sure how big you scale though, it does look like you're still leaving a bit of space around the edges, maybe it'd be fine.

All of these numbers assume the TV is 4:3, not widescreen. The pixels are obviously much wider when displayed at 16:9. With the video encoder width at 640:

(16/9)/(4/3)*(10/11)
=1.212121..

Switching to the 704-wide video encoder setting means:

(16/9)/(4/3)*(1/1)
=1.333333...

That's pixels that are 4*3 rectangles. This is a bit more difficult to work with in terms of obtaining the 1:1 pixels we want for the GBA. I don't know what you're doing right now, but I guess you'd just do the same, something like multiplying the horizontal resolution by 0.75 to get the pixels back to square again. This is obviously a non-integer scale though, but I assume it's something you're already doing anyway, so *shrug*.

EDIT: On a related note, it might be nice if mGBA Wii allowed the user to select what aspect ratio their TV is running in, independently of the Wii's own setting. Personally, I always have my Wii in widescreen, so getting the true pixel-perfect mode requires me to go change the Wii settings just for mGBA.

EDIT2: Now, another option I talked about in that thread is to do a 2*3 (480*480) integer scale, and make up the additional integer with some tricky framebuffer and video encoder size maneuvering—the Wii framebuffer is only 640 pixels wide, of course, so we can't do a true 3*3 (720*480) scale.

My solution in RetroArch was to use a framebuffer width of 530 and a video encoder width of 656. This results in a picture where the pixels are 1.00019058509625:1, i.e. so close to square that it doesn't matter. But let's be douchebags for a minute and say it does matter. I only chose that framebuffer width because it was one that was already built into RetroArch. If we're choosing at will, what's the absolute best framebuffer/vencoder width combo to get us the squarest pixels in town in widescreen?

The Answer
Framebuffer width: 522
Video encoder width: 646
Pixel aspect ratio: 1.00003870118813:1.

The math to hit this is: (10/11) * (640/522) * (2/3) * (4/3) * (646/640) = 1.00003870118813

Again, this is a full 480-line display, so any user who loses some of that to overscan won't be able to see the full picture. Whether it's a mode worth implementing is questionable, since many users won't be able to use it and see everything that's going on. Apparently, though, on the Wii U, overscan in the vWii is nil when viewed over HDMI or on the GamePad. So it might be nice for those users, at the least.

Either way, this is the closest to a 3* integer scale that the Wii can ever hope for. I've played a few games with the RetroArch equivalent to this (even with some picture lost to overscan) and find it really enjoyable, personally. Obviously up to you whether it's worth the effort, endrift.

EDIT3: This is a (line-accurate) approximation of what games look like factoring in overscan on my own TV. And this is a game which really makes use of the very edges, tons of games don't:

Personally, the tradeoff is acceptable.

Are there any plans to implement the 704 video encoder width, or some option to modify the value manually? This would be great news for us Wii users.

@vaguerant Just posted some fixes to the Wii version, can you update and make a list of things I'd need to fix to close this?

its nice that these emulators have video width scaling but is this possible for playing wii games? gamecube games have this feature via Nintendont but I can't seem to find any way to do this for wii games...

@endrift Sorry for the massive delay.

The things that are missing depend on how far you want to go in terms of the stupid nonsense in my previous comment.

I don't have any of my debugging stuff currently and I'm too incompetent to read code, so I'm purely eyeballing this, but it looks like mGBA runs with the framebuffer and video encoder both set to 640 pixels, which will result in a squished (thin/tall) image, horizontally, because of the Wii and GameCube's natural non-square, 10:11 pixels.

A game like Mario Kart: Super Circuit is good for quickly judging ratios because the game's menus are made almost entirely out of circles. Unscientific ruler math (I measured the image on my TV by hand) would seem to confirm that we're almost certainly looking at pixels that are 10:11.

If you just want to achieve square pixels and assuming my estimate of the framebuffer width is accurate, the quickest and cleanest fix would be to simply set the video encoder width to 704. You can probably stop reading this comment right here and close the issue.

Video widths in question

For anybody still reading, I'm gonna quickly get into the various widths we're looking at here and I apologize if it comes across as patronizing, I just want to get across all the terminology I'm (mis)using in this comment.

There are four different widths at play in mGBA on the Wii/GameCube:

1. GBA image width - 240 square, 1:1 pixels--in practice, probably scaled to at least 480 pixels in mGBA by any user of sound mind
2. GameCube/Wii framebuffer width - currently 640, can be between 512 and 640 ratio-agnostic pixels (these are just data points, they have no sense of aspect ratios)
3. GameCube/Wii video encoder width - currently 640, can be between 640 and 720 10:11 pixels (framebuffer is displayed over this many pixels of the ...)
4. GameCube/Wii video signal width - always 720 rectangular, 10:11 pixels (this is the raw video signal sent to the TV for display to the user--the extreme edges are not displayed at all, but they nominally exist)

The last one of these is what causes all our troubles really, the GameCube/Wii natively has these thin/tall pixels, so if we want square pixels in software running on these platforms, we have to make up the width by changing one of the first three.

Methods of correcting the pixel widths to square

Let's assume mGBA is running in the pixel-accurate screen mode with pixelated filtering, for a GBA image that's 480*360. The GameCube/Wii framebuffer is set at 640*480, the GameCube/Wii video encoder width at 640.

This is the current setup as far as I know. The GBA image is now displayed across 480 pixels in the center of the 720 pixel-wide video signal coming from the Wii, with its 10:11 shaped pixels. This makes the GBA pixels, and so the entire screen, thinner than they would be on a real GBA or on any normal platforms's emulator, with square pixels.

You could make up the difference in one of three different ways:

1. by horizontally scaling the GBA image to 528*480 (presumably requiring a bilinear filter to hide the non-integer scale). For a more generalized approach, this is just multiplying the display width by 11/10ths. In most (all?) scenarios, mGBA has plenty of room in the framebuffer to do this (160 unused pixels horizontally at 480*360), but it's kind of a high-level solution to a low-level problem and forces you to use a bilinear filter
2. by adjusting the GameCube/Wii framebuffer width to, say, 582*480. Remember that the video encoder will still display those 582 pixels over 640 pixels, resulting in an apparent pixel width that's almost exactly square (3200:3201). This is an OK solution, but it sacrifices a solid chunk of the framebuffer to achieve it. Sure, the emulated game screen wasn't using it anyway, but the GUI was.
3. by adjusting the video encoder width to display those 640 pixels across 704 pixels of the video signal, resulting in an apparent pixel width that is exactly square (1:1). Framebuffer is completely unchanged.

I would strongly suggest that the last of these is the best option. The video encoder's scaling is very clean, and this avoids doing any weird framebuffer widths or scaling to odd non-integer values within the emulator itself, and it's basically a set-it-and-forget-it approach which changes almost nothing about the way the emulator and its UI run. It's a single "fix everything" button.

This marks the end of everything in this comment I would consider to be a high priority for mGBA for Wii/GameCube, which would allow closing this issue as it resolves everything in the issue as it was filed.

What about that widescreen nonsense?

In my previous comment, I also talked about the possibility of a special widescreen mode for the Wii/GameCube port. This is the kind of stuff you can very freely ignore because it's just one person (me) going "I'd like this" that nobody else is really asking for.

All of the discussion above concerns how mGBA displays on 4:3 TVs (or in 4:3 mode on widescreen TVs). Obviously, there's no room in the framebuffer for a 3* scale of the GBA screen, at 720*480. Also, 4:3 TVs don't have the aspect ratio to display it even if the framebuffer could fit it.

The trick you can do on a widescreen TV though, is to stop targeting 1:1 square pixels entirely, and instead start targeting 0.75:1 pixels. Anamorphic widescreen will do the rest of the work for us from there, you can target 0.75:1 pixels and when stretched by the TV, the result will be 1:1 pixels.

How do we get there?

1. start with the original GBA image - 1:1 pixels
2. scale the GBA image by 2*3 to a resolution of 480*480 - 2:3 (~0.666:1) pixels
3. account for GameCube/Wii's 10:11 pixels - 20:33 (~0.606:1) pixels

Let's pause here for a moment. We've currently got 20:33 pixels, or ~0.606:1. We're well short of 0.75 right now, and even if we push the video encoder way out to the maximum 720 pixels wide, we'd only make it up to 15:22, or ~0.682:1. We're targeting 3:4 or 0:75:1, so we need another push from somewhere else.

The GameCube/Wii video encoder can't display any wider, so we need to come from the other direction: make the framebuffer width smaller so that the video encoder is relatively scaling it up more to make up that last 10% or so of pixel width we need.

4. set the framebuffer width to 522 and video encoder width to 646 (multiply our current pixels by 646/522) - 6460:8613 (~0.75:1) pixels--we're getting into very tiny fractions, that's technically (0.750029025891095:1)
5. have user's TV scale the picture to widescreen (4*3) - 25840:25839 pixels (1.000038701188126:1 pixels)

The result there is pixels that are square to within a tolerance of one 25839th of a pixel. I suspect differences in TVs and even Wiis at this point would have introduce a higher margin of error. The full equation there ultimately being: 1/1 * 2/3 * 10/11 * 646/522 * 4/3 = 1.000038701188126

Now, is 522 framebuffer and 646 video encoder the only option? Not really, it's just the squarest option. I know I said all those things about sacrificing framebuffer width earlier, but here I go suggesting exactly that. It's a while since I worked out the math on it, but there's probably any number of "good enough" combinations that preserve more of the framebuffer.

If you wanted to stick with the video encoder width of 704 at all times, you could try a framebuffer width of 568 in this special widescreen mode, for a pixel width of 640:639, or ~1.002:1. I wouldn't recommend going any higher than 704 in the video encoder, as you'd risk going into overscan, so that (an extra 46 pixels over my "optimal" suggestion) is probably about the most you could save in the framebuffer. This mostly matters for the sake of how much room the mGBA UI needs to remain legible.

I have no real opinion about this; if you did decide to implement a feature like this and you preferred the extra framebuffer room, that seems like a perfectly sensible approach and I doubt the difference of a few hundredths of a pixel width would be noticeable to anybody.

Executive summary

First off, set the video encoder width to 704 as a baseline setting. If square pixels are the entire goal here, you're done.

If you want to look at the nonsense stuff, I think the most logical way to do this in the UI if it were to be added would be to add it as a third option under "Screen mode". "Pixel-Accurate", "Stretched", and ... a better name than "Full Widescreen Pixel-Accurate".

When switching to Full Widescreen Pixel-Accurate, the framebuffer width would drop (to, e.g. 522 or 568) and the video encoder width would also be set to match that setting (e.g. 646 video encoder for 522 framebuffer, or simply remaining at 704 video encoder if you chose 568 framebuffer), the scaling would be forced to 480*480, and the "Horizontal stretch" and "Vertical stretch" settings would probably just be ignored entirely, in the same way that some of them are currently ignored in Pixel-Accurate mode. The "Filtering" setting would continue to function as normal.

Sorry this is so unreasonably long. I hope I got across everything I was trying to say.

@vaguerant You're confusing the video encoder with the video interface, and it can display wider. I've done up to 1440px, and framebuffers up to 960px. Clipping starts misbehaving at these resolutions.

Oh wow, that's huge. Thanks for the correction, @Extrems. I think on the whole you could be a lot more helpful on this subject than me if you're interested/willing. Most of what I "know" about the subject is bad guessing and trial and error; your expertise in this area would certainly be appreciated.

It's interesting you mentioned mGBA for GameCube when the only GameCube port doesn't have this issue.
I'm certainly willing to give pointers to endrift on IRC/Discord when asked.

I unfortunately have no experience with the GameCube port, I just foolishly assumed it shared the same issue because of its similar (identical?) video hardware. Sorry about that. I hadn't realized that port was yours; I would certainly have known you'd gotten it right otherwise.

I tried setting viWidth to 704 (and viXOrigin to 8) but it ended up with clipping on the edges (that might just be my TV), and for some reason on my TV (in 4:3 mode) gave me ~20:21 pixels instead of 1:1. Not entirely sure what the best way to fix that is.