Retailers are expected to hire fewer seasonal workers this year as tariffs and inflationary pressures roil the economy.