The hosts went two goals ahead early on, courtesy of Calvin Andrew and Jimmy McNulty, but Alex Pascanu's strike before half-time was added upon by Ryan Loft in the second period.
Both sides went close late on before a tight penalty shootout - in which substitute goalkeeper Rhys Davies saved twice - allowing Lamine Kaba Sherif to win it 4-3 for the Foxes.
As a result, the Club's youngsters currently sit second in Northern Group B, but need last-placed Fleetwood Town to defeat Bury by either a 1-0 or 2-1 scoreline next Tuesday evening.
City's line-up was a youthful one and included just a solitary change from their 3-1 reverse at Liverpool on Saturday, as attacker Liandro Martis replaced Yohan Benalouane to make his first start of the season.
Before kick-off, Rochdale’s captain, Daniel Adshead, presented the Club with flowers in tribute to our late Chairman, Vichai Srivaddhanaprabha, and those who sadly lost their lives on Saturday 27 October.
Within seconds of the referee's whistle in Greater Manchester, meanwhile, Curaçaoan winger Martis galloped down the flank and crossed quickly for Loft, but the towering forward’s header drifted wide.
However, the Dale, who sit 16th in Sky Bet League 1 following a 0-0 draw at Luton Town on Saturday, took the lead after just five minutes.
The goal came as former Crystal Palace striker Andrew, the hosts' No.9, rose highest to meet Adshead’s corner and send a powerful header beyond Viktor Johansson in the Foxes net.
It was 2-0 to the hosts on 17 minutes. As City's defenders attempted to clear their lines, McNulty’s header evaded Johansson and bobbled into an empty net.
City’s youngsters narrowed the arrears, however, just before the half-hour mark as Pascanu’s brilliantly fearsome drive - from the edge of the Rochdale area - found its way through a full 18-yard box.
As the first half came to a close, home midfielder David Perkins took advantage of an opening in the middle and went for goal, but his ambitious attempt from 25 yards was narrowly over.
After the restart, where Davies replaced Johansson in goal, the Foxes so nearly levelled as Loft muscled his way into the box before sending a low effort just wide.
Leicester, though, were much-improved in the second half and also handed outings to some of the Club's brightest talents.
In the 56th minute, Senior Professional Development Phase Coach, Steve Beaglehole, elected to introduce Luke Thomas, a star of the Under-18s side, to the action in Rochdale.
There was also a Development Squad debut for 15-year-old forward Will Russ, who took the place of Martis in an attacking-three alongside Loft and Callum Wright.
The visitors' youthful exuberance paid off in the 66th minute as Loft found the bottom corner with a low shot at the home side's goal, making it 2-2 and destined for penalties.
Leicester pressed for a goal which would have seen them take all three points, but Kaba Sherif's effort came back off the base of Norman's left-sided post.
Shortly after, Loft stepped up after Russ was fouled in the area to take a spot-kick from 12 yards. However, Loft, too, was denied by the woodwork.
Rochdale then needed the industrious efforts of MJ Williams to bundle the ball off the line, following Josh Knight's attempt at goal, to keep them in the game.
The third-tier hosts almost sealed victory late on, though, as Zach Clough raced clear down the left flank and forced Davies into a smart low stop at his near post.
With the match going to penalties, Davies stepped up again to deny Clough - this time from 12 yards - as well as Williams, while Florent Hoti missed the target altogether.
While Darnell Johnson and Loft saw efforts saved by Norman, Kiernan Dewsbury-Hall, Pascanu, Thomas and finally Kaba Sherif kept their cool to convert, securing a second point.
That means that City still have a chance of reaching the knockout phase. Currently second in Northern Group B, on four points, they need last-place Fleetwood Town to defeat Bury either 1-0 or 2-1 next week.
Any other result would see either Bury (three points) or Fleetwood (one point) secure the second qualification berth. The two sides will meet next Tuesday at 7:45pm at Bury's Gigg Lane.
City: Johansson (Davies 45), Knight, Johnson (c), Pascanu, Ughelumba, Kaba Sherif, Dewsbury-Hall, Leshabela, Loft, Wright (L. Thomas 56), Martis (Russ 62)
Subs not used: Tee, Elewa-Ikpakwu, Husek, Tavares,
Goals: Pascanu 28, Loft 66
Attendance: 615