If there’s one thing President Donald Trump is unquestionably known for, it’s speaking his mind — including about members of his own family.

According to an upcoming book — Kushner, Inc. by journalist Vicki Ward — Trump’s affection for the New England Patriots’ quarterback Tom Brady had him poking fun at his son-in-law’s expense. As reported by the New York Times, Ward claims in the book that while Trump was fond of Jared Kushner, his daughter Ivanka Trump’s husband, he would sometimes quip that Brady could have joined the family instead.

“[Trump] would joke that Ms. Trump could have married Tom Brady,” Ward claimed in her book, according to the Times, “and once joked to Robert K. Kraft, the team’s owner, that ‘Jared is half the size of Tom Brady’s forearm.’ “

Before his presidency, Trump was good pals with Brady (it is unclear what their relationship is like today) and the two routinely shared the golf course together. Brady, who is married to supermodel Gisele Bündchen, was even seen with a “Make America Great Again” hat in his locker during Trump’s campaign.

This isn’t the first time that Trump has commented about a hypothetical relationship between his daughter and Brady. In July 2018, the Times reported that Trump joked “several times” that Brady could have been his son-in-law.

“Instead, I got Jared Kushner,” Trump supposedly said.

In response to the claims in the upcoming book — out March 19 — the White House called it “fiction.”

“It’s sad, but not surprising, the media would spend time promoting a book based on shady anonymous sources and false information instead of all the incredible work Jared and Ivanka are doing for the country,” White House Press Secretary Sarah Sanders said in a statement to PEOPLE.

“The author, on her own website, listed this book in the category of ‘fiction’ — until recently changing it,” the statement continued. “Her initial representation was accurate.”

The White House did not immediately respond to PEOPLE’s request for comment about the Brady comments, specifically.

RELATED: From His Hatred for Coffee to His Early Bedtime: 9 Things You Didn’t Know About Tom Brady

Brady isn’t the only member of the Patriots franchise that the president has been linked to — Trump has been longtime friends with team owner Robert Kraft. Kraft recently became embroiled in controversy when he was charged with two counts of soliciting prostitution in connection with a human trafficking sting tied to a chain of Florida massage parlors, just weeks after New England won their sixth championship under his stewardship.