Edward Latrobe Bateman was a cousin of Governor Latrobe, the grantee of crown allotment 9A Fingal and painted a picture (available online*) of the original Barker homestead near Cape Schanck. He is acknowledged as the architect of Professor Hearn's "Heronswood" at Dromana but doubt has been expressed that he designed "Barragunda" at Cape Schanck- with a concession made that he might have designed the interior. The following statement obviously resulted from a conversation with Robert Stirling Anderson of Barragunda soon after the house was built and leaves no doubt about who designed it.
* ORIGINAL CAPE SCHANCK HOMESTEAD
As pointed out in 1902, the second Barker homestead near Flinders, became Clondrisse.
COOKE'S CLONDRISSE

Illustrated Australian News for Home Readers Saturday 20 May 1871 p 103 Article
Others visit Mr. Anderson in his beautiful Gothic home at Burraguntha, (Barragunda) where they are hospitably entertained by the worthy host and his lady, and after dinner shown through the grounds and garden. The house is truly a pretty place, fashioned from the limestone of the district — a gem of E.L. Bateman's own creation.
KANGAROOING AND CAPE SCHANCK

The Fingal Parish map
shows that much of the land near Cape Schanck was granted to Godfrey Howitt by mid 1866, the year in which Robert Anderson married his daughter, just five years before the above hunt took place.

EventMarriage Event registration number2825 Registration year1866
Personal information
Family nameANDERSON Given namesRobert SexUnknown Spouse's family nameHOWITT Spouse's given namesEdith Mary

ANDERSON—HOWITT.-On the 11th inst., at the residence of the bride's father, by the Rev. Mr. Hetherington, Robert Anderson, Cape Schanck, son of Thomas Anderson, Esq., Knowhead, Ayrshire, to Edith Mary, only daughter of Godfrey Howitt, Esq., M.D., Melbourne. No cards. (P.4, Argus,13-8-1866.)

Barragunda was first mentioned in 1867.
ANDERSON.—On the 17th inst., at Barragunda, Cape Schanck, the wife of Robert Anderson, Esq., of a daughter.
The Australasian (Melbourne, Vic. : 1864 - 1946) Saturday 22 June 1867 p 28 Family Notices

Although the following is chiefly concerned with Bateman's art and his relationships with the Howitts and Georgiana McCrae, it also seems to agree that Barragunda was "a gem of E.L. Bateman's own creation."

E.L.BATEMAN
To William Howitt, arriving in treeless, dusty Melbourne, his brother Godfrey’s house and garden seemed a tranquil and fragrant oasis. So too was his Cape Schanck house Barragunda, designed and completed by Edward La Trobe Bateman before his departure from the colony in 1869.
(P.4, Argus, 27-6-1867.)

Georgiana McCrae is the source of the claim that Bateman only decorated Barragunda.
Victorian Heritage Register VHR H0615

In attempting to discover what Georgiana had actually written about Bateman's involvement, I stumbled upon this fantastic thesis, which although it doesn't focus on the mansion's design, has far more detail about the Howitts, Bateman etc. than appears in biographies and a huge number of illustrations of the art produced at Cape Schanck and Godfrey Howitt's house and garden in Melbourne. Bateman's return home was due to his injured hand.
THESIS.

It was fortunate that the Howitts discarded the Quaker distaste for "Papish" Art and Robert Anderson continued their fostering of it.