Though perhaps not put on a pedestal as high as some more famous songwriting partnerships, The Go-Betweens’ Grant McLennan and Robert Forster produced a catalog of songs as fine as any in rock’s hallowed halls. The two represented a dichotomy of heart (McLennan) and head (Forster) meshing together in perfect harmony. The records they made together ring with the kind of ardency that only comes with tapping into the deepest of artistic wells, which, combined with innate songwriting smarts, created truly timeless results.

After an erratic start (their 1982 debut album, Send Me a Lullaby), McLennan and Forster hit their stride with their second album, 1983’s Before Hollywood, after bassist Robert Vickers solidified the rhythm section with drummer Lindy Morrison. The heavy hand of producer John Brand and some lack of inspiration resulted in a decidedly mediocre third record, Spring Hill Fair, the next year, but fortunately, their best work was still to come.

Reissued last week on vinyl and CD, Liberty Belle and the Black Diamond Express, Tallulah, and 16 Lovers Lane represent the band’s years on the Beggars Banquet label, and between them, contain many of The Go-Betweens’ finest moments. On these records everything seemed to coalesce—pop aspirations, soul-laid-bare lyricism, instrumental deftness—so that when they broke up in 1989, they had definitively left their mark.

For 1986’s Liberty Belle, the band, learning from past mistakes, produced themselves with engineer Richard Preston. It leads off with one of the band’s most recognizable songs, Forster’s “Spring Rain,” but it’s just the beginning of an album that’s spellbinding from start to finish. “The Wrong Road” takes a page from Leonard Cohen, with McLennan delivering poetic reflections on life’s mistakes between swells of strings. On “In the Core of a Flame,” he’s filled with more fervor, demanding “I know what I mean but just don’t say, cos all the old words get in the way. I’ll stay with ‘love’ cos that’s the right word,” over Vickers’ bass punctuation. Forster has a turn on “Head Full of Steam,” his abstruse lyrics combining with ebullient guitar work and backing vocals from Everything But the Girl’s Tracey Thorn for another classic.

Multi-instrumentalist Amanda Brown joined the fold for 1987’s Tallulah, adding touches of violin, oboe, and keyboards, as well as backing vocals, that further emphasized the rich tapestry of McLennan and Forster’s songs. ”Right Here,” which begins the record, is another McLennan song that shines with pop hooks while simultaneously pulling at your heartstrings all the while. Brown makes her presence felt on “The House that Jack Kerouac Built,” her violin leading the way as Forster again shows his ability to turn a phrase, painting a haunting picture as he pleads, “Keep me away from her.” Not to be outdone, McLennan delivers the amazing “Bye Bye Pride.” With lyrics like, “I didn’t know someone could be so lonesome, didn’t know a heart could be tied up and held for ransom,” juxtaposed with a beautiful melody and Brown’s oboe, it traipses the line between bitterness and sweetness.

Released in 1988, 16 Lovers Lane would be the last album of The Go-Betweens’ initial run. (McLennan and Forster reunited in 2000, releasing three more records before Grant’s death in 2006.) It’s hard to imagine a stronger closing statement. The album leads off with one of the band’s best, “Love Goes On!” Here again, McLennan delivers lines that are achingly poignant to anyone who’s ever had their heart broken, revealing he still thinks about her before the song’s seemingly cheerful chorus of “Bah bah bah… love goes on anyway.” “Quiet Heart” does without the cheerfulness, Brown’s plaintive violin accenting lines like, “I tried to tell you… how I miss your quiet, quiet heart.” It’s not all wallowing, though. Forster’s “You Can’t Say No Forever” isn’t exactly sunshine and rainbows, but its criss-crossing guitar lines feel more upbeat. The record also features “Streets of Your Town.” The closest the band ever got to a hit, it pairs effervescent guitar with Brown and McLennan’s voices melting together. “Was There Anything I Could Do?” is another stunner, manic guitar strumming and violin matching the desperation of McLennan’s vocals.

In addition to the three studio albums, Beggars has also reissued Fountains of Youth (Live at the Town and Country Club), an 18-track live album that was previously only released as part of a boxset. The performance was recorded to two-inch tape using a 24-track console in London in 1987, and Forster has admitted that, knowing they were going to be professionally recording enough material for a double album, they took the opportunity to go rogue with the setlist. So while the album features a handful of songs from the still-to-be-released Tallulah, there is a conspicuous absence of cuts from Before Hollywood (no “Cattle and Cane” or “That Way”) and just a few from Liberty Belle. Instead, they favor early obscurities and B-sides, which are an interesting listen to be sure, but nonetheless a bit wanting. What might have been a definitive live set from a band at its height unfortunately falls short. But that takes nothing away from this body of work that should be continuously and forever appreciated.