Welcome to the site !!
Does the shifter itself move freely up/down , and return to center easily ??? , also , it is still possible the fork is bent and/or the shift fork shaft (the shaft that the shift fork slides on to) , lay it on a confirmed flat surface and roll it to see if its bent , even a very slight bend can cause issues , but i would still suspect the fork is bent , since your shifter itself is bent , then when the bike fell on the shifter it puts excessive force on the fork and usually it results in it getting jammed and bent
You can set the fork on a flat surface and with the shaft hole facing up/down look at and measure the distance from the left and right side of each fork tong , they should be equal distance from the flat surface , if not this would indicate the fork is twisted/bent and will need replacement , also look at the shift detent on the clutch side and make sure it is centered and nothing is binding with it , as sometimes the spring becomes jammed or comes off and holds pressure on the shifter shaft which goes all the way thru the transmission to the clutch side
But generally when you have a sufficient impact to the shifter itself , it usually results in a bent fork because the shifter is forced into over extension and the fork is the weak link after the shifter itself , and your symptoms seem to be confirming that , even the slightest tweak will make shifting difficult ......