Product development takes teamwork, math, and smaller boxes
Frequently once engineers solve a particular problem, no one bothers going back to look at whether there might be other or even better ways of solving it. The assumption is that since the solution works, there’s really no need to revisit it, and besides there are always new problems to be addressed.
Such was the case with the synchronous time interleaving technology that has been used for signal acquisition in oscilloscopes since the 1970s. One of the attractions to synchronous time interleaving is the simple and straightforward mathematical model behind its operation. This elegance contributed to its staying power, such that it was considered the best way to go and no one bothered to look any further. While developing the DPO70000SX series oscilloscopes (Figure 1), Tektronix engineers had to devise new approaches that challenges long-held assumptions.
Figure 1. Although the box conforms to 3U height, the DPO70000SX series oscilloscope offers 70 GHz bandwidth.
Take the story of Dan Knierim (Figure 2) and the discovery of asynchronous time interleaving. Knierim is a math guy. Some would consider him a math genius. His parents were both technologists and passed on the ability to think in mathematical formulas to their sons. As kids, Knierim and his brother David would solve math problems in their heads before going to sleep. Now he solves more complex problems for Tektronix where he has earned 39 patents in the past 33 years.
Figure 2. Dan Knierim proved that there were ways around the traditional synhronous pre-sampling.
Knierim needed to devise ways to improve oscilloscope performance. Going back to synchronous time interleaving, one late night in the office fueled by curiosity and a bit of chocolate, Knierim wondered if there were other solutions. "Why does the pre-sampler have to be synchronous with the downstream analog-to-digital converter?" he asked.
The breakthrough happened when he decided to "pull out an old trick" and prove that there were no low-noise alternatives to synchronous time interleaving. When he was unable to find the proof, he realized that despite the elegance and simplicity of synchronous pre-sampling of signals, other approaches were possible.
Knierim’s conclusion opened the door to exploring new ways of solving the problem, specifically asynchronous pre-sampling. Over the course of the evening, he realized that this approach would not only boost performance over a synchronous approach, but that it would also maintain the low-noise advantage of time interleaving. "The math started working and everything started to fall into place," he said. "By the end of evening I had the basic math figured out." The next step was to see if the late-night math would actually work. Knierim and colleague Jim Lamb then put a circuit simulation together demonstrating that not only was it possible, but it could be built with existing technology.
Part of what made this possible was earlier ground-breaking work by fellow engineers Kan Tan and John Pickerd. In 2003, they identified the value of using high-bandwidth mixers in front of existing oscilloscope digitizers. The significance of this technology is that the most expensive-to-develop ICs in a digitizer could be used with less costly front-end mixer components to increase overall performance. This lets the system achieve higher bandwidth and sample rate without significantly increasing costs.
Knierim’s discovery ultimately led to a patent award for ATI (Asynchronous Time Interleaving) while Pickerd and Tan were awarded a patent on their broader general mixer overlay invention. Now fully implemented and functional, ATI is a key enabling technology behind the Tektronix DPO70000SX oscilloscope family. It extends real time oscilloscope bandwidth to 70 GHz with a 200 GS/s sample rate along with high signal-to-noise ratio (SNR).
Coherent optical drives
With coherent optical applications driving data speeds from 100 Gbps to 400 Gbps and soon to 1 Tbps, oscilloscope bandwidth requirements are on the upswing. "Where 33 GHz was the upper limit for most applications up until recently, we’re now seeing demand for bandwidth around 70 GHz for optical communication as well as for scientific research applications," said Mike Wadzita, senior product planner for Tektronix.
Wadzita learned that oscilloscope users also wanted the shortest path possible from the device under test (DUT) to the oscilloscopes input. Scalability and versatility were also major considerations.
The way people use oscilloscopes is also changing. The industry trend had been toward bigger and bigger boxes with larger screen sizes, but that was no longer the case. "Customers told me they can easily buy a much bigger monitor than anything we could put on an oscilloscope," Wadzita related. "All they are really interested in is signal fidelity of the acquisition system and being able to access waveform data remotely."
Based on this input, Wadzita mapped out a new direction for the oscilloscope. Instead of a bigger footprint, the oscilloscope would shrink to fit in a standard 3U rack slot, meaning it needed to be exactly 5¼-in. tall. To meet scalability and flexibility requirements, it would only have one high-speed channel, but because coherent optical tests require four synchronous channels, there would need to be some way to precisely synchronize up to four oscilloscopes to behave as a single unit.
"Scalability isn’t just about having more channels," Wadzita explained. "It’s being able to scale down so that more people can take advantage of the investment in a high-performance oscilloscope. Usage patterns are changing and equipment doesn’t sit around. It gets aggregated in a lab when needed for a major project and distributed out to other users when the project is completed."
Could it even be built? Would ATI deliver the necessary performance gains? Could the internals of an already crowded oscilloscope be packed into a 3U-size package? Could the timing and clock synchronization questions be solved while meeting performance and usability requirements? Could it be developed in a reasonable amount of time and budget with existing resources and technologies?
More questions than answers
Once management gave the green light, a broad cross-functional team of hardware, software and mechanical engineers was assembled. Project Tabasco, the internal code name, started with many more questions than there were answers.
One of the big questions revolved around figuring out how to synchronize multiple oscilloscopes in a way that would meet customer needs around scalability and flexibility. The team had some experience connecting multiple instruments, including a previous generation analog method that could be used to trigger multiple scopes and connect scopes and logic analyzers. This system, however, lacked the performance needed for the Tabasco project and was not a viable option.
Fortunately, there was a viable solution waiting in the wings. Bart Hickman, one of the main architects of what would ultimately be known as the UltraSync interface, had previously devised a way to synchronize oscilloscopes, but it was never implemented due to a lack of market requirement.
"The basis for UltraSync was just an out-of-the-blue proposal I made well before this project to simply extend the internal digital synchronization of our existing scopes so that it could go between multiple scopes," said Hickman. "I refined the idea to handle oscilloscopes with different internal control architectures so future scopes could work with present-day instruments. I reviewed the idea with other engineers and we all agreed it was a great way to do it—and then the idea was basically shelved."
Given the opportunity to move forward, Hickman together with designers Jed Andrews and Jeff Mucha dusted off the earlier proposal and began experiments that isolated problems around passing clock information across units, managing temperature drift and maintaining timing relationships.
After narrowing it down to three possible configurations, the design team opted for a ring oscillator configuration because it was the simplest to implement. The frequency of the ring oscillator reflects the time delay between oscilloscopes, allowing optimization of the synchronous communication and instrument acquisition skew. Control is established by one oscilloscope acting as the master based on the way cables are connected, but either the master or extension can initiate a trigger. Because each oscilloscope is a complete standalone instrument, only processed waveforms are sent over a 5GT/sec PCI Express data link, minimizing network traffic.
Updated "back of the napkin" moment
Mucha, who also served as the hardware lead for the project, and Marc Gessford, the mechanical lead, faced a significant challenge in trying to shrink a performance oscilloscope to fit into a now much smaller package. The working assumption across most of the team was that it couldn’t be done.
To get the ball rolling, Mucha turned to a simple 3D CAD tool called SketchUp (Figure 3). In the modern equivalent of a "back of napkin" moment, Mucha’s impromptu 3D models of what the internal package might look like opened a lot of eyes: not only did repackaging appear feasible, but it might actually not result in any show-stopping tradeoffs. One thing that was clear, however, is that it would not be possible to use two ATI acquisition boards in a single box due to power requirements, putting even more pressure on the team developing UltraSync.
Figure 3. Hardware designer Jeff Mucha used SketchUp to show how a performance oscilloscope potentially could be housed in a smaller package.
One of the most important considerations in an oscilloscope is maintaining a stable operating temperature. The mechanical designers, therefore, focused their attention on maintaining consistent air flow throughout the box, while the hardware team designed in an array of temperature sensors on the ASICs, on the acquisition board, and on each of the ADCs (analog-to-digital converters). In the end, the new compact design ran a few degrees cooler than traditionally packaged ultra high-performance oscilloscopes.
A critical inflection point came when Mucha and mechanical designer Marc Gessford happened to run into each other while walking across the Tektronix campus. The key problem was that re-using an existing relay tray design would mean rotating the acquisition board (Figure 4), which in turn would result in a longer signal path. But redesigning the part could also pose problems. During this quick interaction, the decision was made: redesign it. "Fortunately it wasn’t too hard and it definitely was the right thing to do," said Mucha.
Figure 4. Mechanical designer Marc Gessford (left) highlights details of the new ATI acquisition board for colleagues.
The DPO70000SX oscilloscope incorporates 16 new circuit boards and three new ASICs, which involved two full-board spins. "It was a great moment when we first powered up the prototype and it didn’t catch fire. It just worked. It was an amazing feeling when the LED came on showing an acquisition," Mucha recalls. Figure 5 shows prototype boards undergoing tests.
Figure 5. Prototype boards and acquisition systems undergo testing in Tektronix labs.
A stabilizing influence throughout the mechanical and hardware design process, Wadzita refused to budge in the belief that 3U was what customers wanted. "They said it wasn’t going to be possible at 5.25 inches and wanted to add another inch or so in height. It was pretty clear in what I heard from customers that it had to be 3U. The end result is something that is right for customers and we all can be proud of making it happen."
A software Scrum
With new acquisition technology, multi-box synchronization, three new ASICs, and redesigned hardware, the software team had its hands full. Eric Thums, the software lead for the Tabasco project, says that it was "the most complex and challenging project that succeeded" that he has worked on over the course of his career.
Given all the moving parts on the project, the Scrum software development methodology widely used within Tektronix proved to be a good fit. Scrum, a type of agile development methodology, is analogous to jazz music where you know the start and end, but what happens in between is fluid, determined by whatever is highest priority at the moment.
A key aspect to Scrum is a dedication to planning so that everyone on the team has a clear understanding of the entire project and can step in when and where they are likely to be needed most. Since there isn’t a lot of information about using Scrum on hardware-centric projects, according to Thums, the team adopted a cautious approach with 15 engineers participating in more than 30 hours of up-front planning.
Sponsor video, mouseover for sound
While the software may not have yielded many breakthroughs, it did involve what software engineer Ed Tanous called a lot of "wicked problems." These are the problems that you really have no idea how to solve until you solve them.
One such problem was managing phase calibration in the UltraSync system down to sub picosecond levels. "This was a very challenging problem that took a lot of learning to understand and I had to ask for a lot of help," explained fellow software engineer Evan Belt. "Ultimately, it came down to solid engineering of getting it down to the smallest pieces possible and stepping through the problem one increment at a time."
Another challenge was maintaining compatibility with Tektronix application software such as DPOJET, SDLA and SignalVu, along with partner applications developed over the years for Tektronix oscilloscopes. Despite the new performance and other innovations, this oscilloscope had to functionally behave like any other Tektronix oscilloscope.
"I’m happy to say that we achieved a lot of goals on this one," said Thums. "It required a lot of cross-discipline teamwork and willingness to set egos aside while keeping the goal in sight."
Lego time
As the project pieces started to come together, Wadzita and Gessford created scale models of the new form factor using wood blocks. The idea was to help the team visualize how the smaller oscilloscopes could be arranged around a device under test (DUT) to minimize cable length. The idea caught on and eventually the wood blocks became Lego blocks (Figure 6) that could be given away to others in the company as well as early adopter customers.
Figure 6. Lego block models helped the design team, other people in the company and customers visualize the size and configuration options provide by the smaller profile design.
Figure 7. Locating the connectors in the center provides for the shortest possible connections between two oscilloscopes when one is mounted upside down.
In discussing the design with customers, one point became clear: the 70 GHz ATI input port needed to be in the bottom-center of the box. This would allow users to stack boxes such that the inputs would be directly on top of each other when one box was inverted or flipped upside down (Figure 7). Originally the port was on the side where flipping would actually move the ports further apart. The miniature wood and Lego blocks helped customers and the team visualize different configurations, such as arranging four units in an "L" shape around the DUT.
Raising the bar
From the outset, we knew that building a new high-performance oscilloscope in half the footprint contained considerable risk. As anyone who has ever worked on projects of similar size, scope and ambition, the only hope for success is through teamwork. Remarkable insights, such as Knierim’s discovery of asynchronous time interleaving, certainly helps. Even so, it takes determined engineering work across the board to put those discoveries into action.
While many of those long hours and late nights may go unnoticed, they will undoubtedly be appreciated by the hardware design engineer who connects a pair these oscilloscopes and gets a clean eye diagram on a 32 Gbps differential signal. That’s when the team will know it was all worthwhile.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
