The poem is certainly funny to read with all its "allitérations" and "contrepèteries" (keeping the sounds but changing their place to change the meaning), more funny if you're looking at you in the mirror or another person saying it with always the same motion of the lips ("m", the other consonant, "r", being invisible),
musical too, but what else ?

(reminds me one of my brother's friends who, in the early eighties, used to answer any convincing (or not convincing) discourse, any adverse political theory, philosophy, by those simple words: "Et alors ?" ("So what ?"). And nobody could answer anything to this "Et alors ?", it was the end of it, the global response)

The thing is, there is also some "classical" poetry in it, Perec is clever, like "mer amarrée à ma rame", which is a beautiful idea: it is not the boat (the oar) which is moored but the sea to the oar. "Âme même amie" is fine too, remembering the french pronounciation sticking the words with no accent at all and producing the strange "Amemèmami" word, producing sounds so strange at hearing you can't give them any meaning even if they have.

And the absent word too: "mère" (mother), same sound as "mer" (sea). He tried fast every word with an "m" and a "r", not that one which is precisely pronounced like the place where he plays his poem and so closely connected to the title "aimer" (love/like). And in "aimer", you have "mer".

Would I dare to say it ? It is the thing that makes me a little reticent about it: too clever. Or: "just a game". And game is central in Perec's world. As a teenager, I was fond of Hugo (Victor), who liked also making "game-poems". You have this "Les Djinns" poem, about wandering spirits:

Murs, ville
Et port,
Asile
De mort,
Mer grise
Où brise
La brise
Tout dort.

And so on, every strophe having one syllable more till the climax of the poem, then one syllable less till the end. Very efficient, you can hear something even if you don't understand anything.
But, if you try to understand, well, so "hugolian": this romantism of ours, quite superficial, only in the "pose", in the picturesque, the colourful.

To go deeper, I prefer the few deep (crazy) spirits we have in store here, like Marceline Desbordes-Valmore, she wrote astonishing poems, like this marvel:

C'est là que j'ai vu Rose Dassonville,
Ce mouvant miroir d'une rose au vent.
Quand ses doux printemps erraient par la ville,
Ils embaumaient l'air libre et triomphant.

Et chacun disait en perçant la foule :
" Quoi ! belle à ce point ?... Je veux voir aussi... "
Et l'enfant passait comme l'eau qui coule
Sans se demander : " Qui voit-on ici ? "

Un souffle effeuilla Rose Dassonville.
Son logis cessa de fleurir la ville,
Et, triste aujourd'hui comme le voilà,
C'est là !

She has the music, and the deep melancholy too.

I think Perec was so found of "sounds", "allitérations" because of the very difficulty it is to write something musical in french, rythmic, french being, since the 17th century, a "flat" language. And you have the two schools in french poetry since long: musical without much meaning (Verlaine, Rimbaud, Appollinaire) or rethoric without much music (Hugo, Beaudelaire). Ce qui est un peu vite dit.

And maybe is it the "native speaker" disease: asking too much of poems written in you language. In other languages, every bit of poem brings more than meaning and music, it brings something you have inside yourself and don't know how to express. Well, music after all.