"The Star-Mouse" (1942) by Fredric Brown. It's available on Project Gutenberg.
It starts:
Mitkey, the mouse, wasn't Mitkey then.
He was just another mouse, who lived behind the floorboards and plaster of the house of the great Herr Professor Oberburger, formerly of Vienna and Heidelberg; then a refugee from the excessive admiration of more powerful of his fellow-countrymen. The excessive admiration had concerned, not Herr Oberburger himself, but a certain gas which had been a by-product of an unsuccessful rocket fuel—which might have been a highly-successful something else.
The aliens say this:
"Exactly. Focused upon this creature's brain-center, they can, without disturbing his memories, be so delicately adjusted as to increase his intelligence—now probably about .0001 in the scale—to the point where he is a reasoning creature. Almost automatically, during the process, he will assimilate his own memories, and understand them just as he would if he had been intelligent at the time he received those impressions.
Back on Earth, Mitkey (who learned English by listening to the German professor) says:
"Yess, Brofessor. I see your boint, that you think an intelligent race of mices und an intelligent race of men couldt nodt get along side by sides. But it vould not be side by sides; as I said, there are only a ferry few beople in the smallest continent of Australia. Und it vould cost little to bring them back und turn offer that continent to us mices. Ve vould call it Moustralia instead Australia, und ve vould instead of Sydney call der capital Dissney, in honor of—"
And it ends with:
It wasn't deliberate. It couldn't have been, because the Professor didn't know about Klarloth's warning to Mitkey about carelessness with electricity—"Der new molecular rearrangement of your brain center—it iss unstable, und—"
And the Professor was still back in the lighted room when Mitkey ran into the room where Minnie was in her barless cage. She was asleep, and the sight of her—Memory of his earlier days came back like a flash and suddenly Mitkey knew how lonesome he had been.
"Minnie!" he called, forgetting that she could not understand.
And stepped up on the board where she lay. "Squeak!" The mild electrical current between the two strips of tinfoil got him.
There was silence for a while.
Then: "Mitkey," called the Herr Professor. "Come on back und ve vill discuss this—"
He stepped through the doorway and saw them, there in the gray light of dawn, two small gray mice cuddled happily together. He couldn't tell which was which, because Mitkey's teeth had torn off the red and yellow garments which had suddenly been strange, confining and obnoxious things.
"Vot on earth?" asked Professor Oberburger. Then he remembered the current, and guessed.
"Mitkey! Can you no longer talk? Iss der—"
Silence.
Then the Professor smiled. "Mitkey," he said, "my little star-mouse. I think you are more happier now."