Hello, there is no window icon displaying for versions prior to 1.6.x.

This is because the pre-1.6 assets index doesn't have an icons directory.

It should be added from the legacy assets index.

Here is the log that states that the icon is missing:

[Info: 2021-06-12 10:49:44.5077274: GameCallbacks.cpp(162)] Game/game () Info Turning of ImageIO disk-caching
[Info: 2021-06-12 10:49:44.5129649: GameCallbacks.cpp(162)] Game/game () Info Loading current icons for window from: C:\Users******\AppData\Roaming\.minecraft\assets\virtual\pre-1.6\icons\icon_16x16.png and C:\Users******\AppData\Roaming\.minecraft\assets\virtual\pre-1.6\icons\icon_32x32.png
[Info: 2021-06-12 10:49:44.5137577: GameCallbacks.cpp(162)] Game/game () Info javax.imageio.IIOException: Can't read input file!
[Info: 2021-06-12 10:49:44.5141002: GameCallbacks.cpp(162)] Game/game () Info at javax.imageio.ImageIO.read(ImageIO.java:1301)
[Info: 2021-06-12 10:49:44.5170612: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.launchwrapper.injector.VanillaTweakInjector.loadIcon(VanillaTweakInjector.java:130)
[Info: 2021-06-12 10:49:44.5171423: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.launchwrapper.injector.VanillaTweakInjector.loadIconsOnFrames(VanillaTweakInjector.java:107)
[Info: 2021-06-12 10:49:44.5171787: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.launchwrapper.injector.VanillaTweakInjector.inject(VanillaTweakInjector.java:94)
[Info: 2021-06-12 10:49:44.5172074: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.client.Minecraft.main(SourceFile:1685)
[Info: 2021-06-12 10:49:44.5172335: GameCallbacks.cpp(162)] Game/game () Info at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[Info: 2021-06-12 10:49:44.5172583: GameCallbacks.cpp(162)] Game/game () Info at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[Info: 2021-06-12 10:49:44.5172848: GameCallbacks.cpp(162)] Game/game () Info at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[Info: 2021-06-12 10:49:44.5173090: GameCallbacks.cpp(162)] Game/game () Info at java.lang.reflect.Method.invoke(Method.java:497)
[Info: 2021-06-12 10:49:44.5173317: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.launchwrapper.Launch.launch(Launch.java:57)
[Info: 2021-06-12 10:49:44.5173561: GameCallbacks.cpp(162)] Game/game () Info at net.minecraft.launchwrapper.Launch.main(Launch.java:18)

The no-icon image shows the missing icon.

Have a nice day !