Hilary Duffis remembering her ex,Aaron Carter, after his death.

Not long after it was confirmed thatCarter had died at the age of 34on Saturday, Duff, 35, mourned the loss and shared her heartbreak over social media.

“I am deeply sorry that life was so hard and that you had to struggle in front of the whole world,” the actress and singer captioned anInstagram post. “You had a charm that was absolutely effervescent.”

Dia Dipasupil/Getty; Gilbert Carrasquillo/WireImage

NEW YORK, NEW YORK - MAY 17: Hilary Duff attends the 2022 ABC Disney Upfront at Basketball City - Pier 36 - South Street on May 17, 2022 in New York City. (Photo by Dia Dipasupil/Getty Images,); PHILADELPHIA, PENNSYLVANIA - APRIL 10: Aaron Carter attends the Celebrity Boxing Face Off between Lamar Odom & Aaron Carter on April 10, 2021 in Philadelphia, Pennsylvania. (Photo by Gilbert Carrasquillo/WireImage)

Duff and Carter dated back in 2001, and the “I Want Candy” musician even made a guest appearance on an episode of Duff’sLizzie McGuire.The pair later broke up in 2004, and Duff, now a mom of three, marriedMatthew Komain 2019.

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

L. Cohen/WireImage

Aaron Carter and Hilary Duff during The Lizzie McGuire Movie-Premiere After Party at The El Capitan Theater in Hollywood, CA, United States. (Photo by L. Cohen/WireImage)

Alongside Duff, Carter’s twin sister,Angel Carter, also used social media as a way ofmourning the loss of her brother.

“To my twin… I loved you beyond measure. You will be missed dearly,” she began. “My funny, sweet Aaron, I have so many memories of you … and I promise to cherish them.”

For more on Aaron Carter, listen below to our daily podcast PEOPLE Every Day.

Angel, 34, added: “I know you’re at peace now. I will carry you with me until the day I die and get to see you again. 💔.”

Vincent Sandoval/WireImage

HOLLYWOOD, CA - JUNE 04: Angel Carter (L) and singer Aaron Carter attend Los Angeles opening night of “Joseph and The Amazing Technicolor Dreamcoat” at the Pantages Theatre on June 4, 2014 in Hollywood, California. (Photo by Vincent Sandoval/WireImage)

Celebritiesacross the entertainment world similarly mourned Carter, the former child star who released a 3X platinum album at just 13 years old, online as well.

“Fame at a young age is often more a curse than a blessing and surviving it is not easy,” songwriter Diane Warren shared on Twitter. “RIP Aaron Carter.”

One Tree Hillactor Tyler Hilton wrote that the news was “heartbreaking” after knowing Carter “for years” and taking a liking to him. Hilton and Carter toured together as part of thePop2K Tourin 2019, and Hilton even playedCarter’s hit “I Want Candy” along with himon stage.

“Known him for years and always really liked him, he was warm and really funny,” Hilton shared on Twitter. “Loved putting on a show and he was good at it. I’ll find some pics of us and post more later… damn RIP budddd.”

“Incredibly sad about the passing of Aaron Carter. My thoughts and prayers go out to his friends and family,“tweetedDisney alum Christy Carlson Romano.

M. Caulfield/WireImage

Aaron Carter and Billy Gilman introduce an act at the 2001 Billboard Awards at the MGM Grand Hotel in Las Vegas. (Photo by M. Caulfield/WireImage)

Want to get the biggest stories fromPEOPLEevery weekday?Subscribe to our new podcast, PEOPLE Every Day, to get the essential celebrity, entertainment and human interest news stories Monday through Friday.

source: people.com