If you want to install Ubuntu in VirtualBox on your Windows 11 computer, make sure to enable Hardware Virtualization from BIOS and then follow this guide.

If you deploy virtual machines with VirtualBox, you'll want to use the shared folder feature. Find out how this is done in VirtualBox 7.

Also looking inside the memtest86-iso.iso file, I don't see anything I recognise as characteristic of bootable media e.g. it doesn't look like a linux bootable or a windows bootable..