PROJECT: Kanon (Kyoto animation) DVD (16/24 released)

[Rockman-X]

The problem here is that you are trying to make a lossy recompress of an already lossy compressed video. No matter what you do, it won't look as good as the original. Kanon 2006 DVDISOs shouldn't be hard to find, considering the series' popularity.

Why don't you just add softsubs in your language? That way you won't need to reencode it, unless you REALLY want to limit file size. Remember: Lower file size = crappier looking encode. I never limit file sizes, instead I opt for quality-based encoding (CRF 16~18 on x264). Always gives great results, but the file sizes are unpredictable.

If you just want to add your subtitles, just open the file on MKVToolnix/MKVMerge GUI, add your subtitles, mux it to a new file. Alternatively, you can even load the subtitles externally, by renaming them with the same filename of the original episode, ie.: "Kanon 02.mkv" and "Kanon 02.ass". This way people can download the original releases and all you need to make is distributing a "subtitle pack".

But of course, if you are, even for a moment, thinking about adding HARDSUBBED subtitles to it, then you won't be getting any help from me.