Caldwell came on late in the season when Wes Welker was sidelined with a concussion. The 28-year-old Caldwell caught 11 of his 16 passes over the final three games of the season.
His most productive game in 2013 was against San Diego on Dec. 12, when he had six receptions for 59 yards and two TDs.
Caldwell was held without a catch in a 43-8 loss to Seattle in the Super Bowl.
Over his six-year career, Caldwell has 141 catches and nine scores. He spent his first four seasons with Cincinnati after the Bengals took him in the third round of the 2008 draft.