PicoBlog

One of the more interesting stories over the last ten years has been the growth of Arm in the server and enterprise markets. Almost a decade ago, Arm was predicting 25%+ of these markets would be enabled through different merchant silicon providers, effectively elbowing out all the x86 install base (which was mostly Intel at the time). Around a dozen start-ups came to the fore, developing silicon for those markets using off-the-shelf Arm mobile cores, with limited success. Due to performance, power, software support, or lack of support, all of those companies dissolved away through acquisition, merger, or simply left the market. Cavium was acquired by Marvell, Calxeda shuttered, Qualcomm mothballed their efforts, Nuvia was acquired by Qualcomm but for smartphones, Annapurna was acquired by Amazon, Huawei has limited manufacturing, and finally AppliedMicro was acquired by Ampere Computing.

Out of that list, Marvell uses its custom silicon team for specific designs, Amazon builds its own, Huawei is trying to adapt to local manufacturing, leaving Ampere as the only one with a merchant silicon option. In this instance, merchant silicon means chips or solutions that anyone can go out and buy. None of the other offerings does this, in volume, globally.

Heading up Ampere Computing from its inception is CEO Renée James. Renée has a detailed history in semiconductors, having spent 28 years at Intel. Her roles in that tenure included a stint as Chief of Staff to Intel co-founder and CEO Andy Grove, heading strategic expansion into software in enterprise as EVP of software and services, managing relationships with all the major leading device companies, product management of the 386 family of products, and COO of Intel’s Online Services, before becoming President of Intel in 2013 in charge of Global Operations and Manufacturing. Renée also was appointed Vice-Chair of the National Security Telecommunications Advisory Committee, and serves/has served on the boards of Vodafone, Oracle, Citigroup, and VMware.

After leaving Intel in 2016, Ampere Computing was founded in 2018, with the goal of building the products that other companies failed to address in the datacentre. This revolved around a more energy efficient way to bring computational performance, and as a result Ampere acquired AppliedMicro, hired a lot of the staff from other failed Arm server start-ups, and a number of other industry chip designers, to figure out what went wrong before and how to bring hardware to the market.

Share

In this interview, I wanted to broach the question of Ampere Computing’s intent – why have they succeed where others have failed, and is the goal of the company still the same given the dynamics of the market have shifted towards machine learning accelerators rather than pure CPU horsepower. You can watch the interview here, or read the transcript below.

This transcript is provided as-is, with minor updates to improve readability.

Ian Cutress: Can you describe what or who Ampere is, and should I be calling it Ampere or Ampere Computing?

Renée James: You can call it either! The reason the company’s name is Ampere Computing is because when we wanted to get all the trademarks and brands and websites and everything, you can imagine Ampere being an actual person’s name in technology in physics, and in France – it is difficult to trademark. So we got Ampere the trademark for the category that we own, even though others have called their products Ampere. We’ve had conversations with those companies.

Ian: Is it fun having those conversations?

Renée: We’re all friends. But Ampere Computing is the name of the company, and we always expected people to just call it Ampere. We also knew, because as you point out I had a lengthy career in semiconductors, that in fact that it's not all going to fit on a chip. God help us if you're going to make something that big (holds up chip), and you wanted to have something iconic, like our A logo. Because in the end, that's really how people identify - with an iconic looking logo.

Ian: When I saw the “A” logo, I was at that time working for AnandTech. It’s not too dissimilar!

Renée: Our “A” is very specific. It is specific in colour, and in style. You may not know this, but whenever you did something wonderful, my late great mentor would send you a little handwritten note, signed with a single A in a red felt tip pen. When you did something great and amazing – he sent me one when I started the next semiconductor company in Silicon Valley.

Ian: He sent you a message [of congratulations] did he?

Renée: He sent many. He was a big fan of me doing this - more of a fan of me doing this than I was a fan of doing it! I kept saying ‘Andy, it’s a lot of money’ and all of those things. Being the founder was not the hardest thing - if you're going to build high-performance silicon, there are so many things that are more difficult. I had had a lot of experience in management, and I had managed 100,000 people. Running a company was not going to be the big challenge, which a lot of founders have issues with - my big thing was that it's going to take a lot of money to do this right.

Ian: For those of the audience who are unaware of what Ampere do, what do you do?

Renée: We build high-performance CPUs. But one of the challenges that we had been unable to really tackle, prior to the starting of Ampere, was the power wall. So what we build are high-performance microprocessors for cloud servers that are environmentally friendly. That is to say, depending on the core count or what the objective of the product is, it is in a very tight power envelope.

The greatest limiter in our [global] ability to continue to do what we’re trying to do with cloud computing and where everyone wants to go, including with AI, is power. In certain jurisdictions, the power is already restricted on the grid. Here in Silicon Valley, in Santa Clara County, you can't build a data centre anymore - it's not a thing, because there is no more power. So we’ve known this for a while. We said if we’re going to go and do something new, especially in an area that I had clearly spent a lot of time in and many of the others that immediately joined the company had been doing, such as super-high performance on the leading edge, we said we’ll go after where we think the world needs to go. This means we deal with what most normal people would call ‘the eco issue’, but it’s really a power challenge.

Ian: When the company was formed, you were quoted as saying that one of the goals of Ampere was to tackle part of the market that Intel wasn't focusing on. Given what you’ve just said about energy and power, and given how the competition has changed since then, is that still true?

Renée: I think it is interesting. It is somewhat true, and here's what I mean. One of the great parts of technology is you have a thesis - if you're an invention company, and here at Ampere we are inventors, you lean into the unknown. You think that it could be different than it is today, where we build 450-500 watt processors. What if we only had 200-250 watts? What would we do differently? But we know that the workloads need more and more performance every year.

You know you're right when everyone copies you. So directionally we think this is the future for sure. I don't think there's any question. The only question is how fast everything will get there. But it seems we’ve all taken a brief hiatus while we all talk about deploying 500 watt to 1 kilowatt graphics cards. Because that’s super eco-friendly!

Ian: There’s a research paper from TSMC about cooling future 2.6 kilowatt chips, and you can tell the industry is going that way. Whether it's the right way is up for debate.

Renée: I think that in technology, you have to allow for the extremes. There are cases where that's going to be needed to get through certain things. We don't build this stuff just for the hell of it, although it is super fun. Who wouldn't want to do this for a living?

But why did I get up in the morning, after 28 years at Intel, and say why did we do this? Why did I then start a company and take this on? Because of what we facilitate. It's about what people can do with that power budget. So do I think we need those super high-end liquid-cooled systems or whatever? Yes, I do. Why? Because there are things that we won't be able to know, problems we won't be able to solve unless we have that. Is that going to be mainstream? I don't think so.

We think we're going to be able to satisfy the growth in the industry, and the underlying demand for computing. If we don't do something that’s mainstream, high-performance, but also cost-effective, and also in a power range that we can deploy broadly and scale out – that’s the problem we’re trying to solve. We aren't trying to solve that other problem, but I do think there are people that need to solve that problem.

Ian: One of the things that Ampere has always put in the messaging since day 1, is the attachment to the Arm ecosystem, developing Arm servers, and the first couple of generations of products you used Arm off the shelf cores, their ‘Neoverse’ line. But for the newest generation “AmpereOne”, you decided to do semi-custom Arm cores. I think that caught a lot of us by surprise because you were seen as the poster child for Arm Neoverse. So, I’d like to poke your brain as to why go semi-custom?

Renée: First of all, it can take 2-4 years to build a new microprocessor microarchitecture. So if you're going to start an investor-backed company of this magnitude, you need a product to sell as fast as possible. I think the Arm Neoverse cores are outstanding, and we've learned a lot working with them. We’re really excited about what we learned - in fact, there were days when we were building Altra, that the chief fellow who was designing it said ‘I really should have thought about this differently in my prior career!’ There really are some incredible benefits to the Arm technology, none of which we are abandoning in any way.

Here is a significant consideration though. When you're building merchant market products, you're building something to be absolutely the best it can be at solving certain problems, or certain classes of problems. At the rate and pace that the industry likes to move, that isn’t always the rate and pace of a generalised roadmap for a broad pace of people to draw off of - it doesn't give you the ability to think very specifically in the microarchitecture about how you're going to solve certain things, offload certain problems, move data, and memory and all those things. Those have been our high order bit issues, for as long as I've been doing this - so that’s why in high-performance computing you ordinarily rely on all these different aspects of your architecture.

It’s not just the core - the core is actually not the only thing you need to worry about. It is one of the main things, but you need to worry about a lot of the other features, and what you can put into it and how do you differentiate. If you're merchant market silicon, you need to differentiate, and it's not just on performance, it has to be on features. Where those features come from - that you then have to have unique design and IPs.

The IPs that go in servers are not well developed in the industry, because in the server companies that have existed, the off the shelf IP don't exist because the two purveyors of x86 do all their own. So if you want to build something that is higher performance than the legacy environment, and does unique things that customers want to pay you for, you can't take an off-the-shelf thing that everyone else can use and be like ‘Oh this is great’ at whatever rate and pace. That rate and pace may be fantastic, I'm not being in any way derogatory against it - I think its fine, but it might not be at the rate and pace you need for your customers. So that had always been our strategy from day one, we just didn't tell anyone.

Ian: When Arm wanted to breach into the server market, there was this big announcement of 25% of Arm silicon in servers in X amount of years. Speaking with them at the time, they were envisioning a market with a dozen merchant market silicon providers that were doing their own thing. To your point, we have had a dozen come and go, but only Ampere is left. Is it because of that end-to-end difficulty?

Renée: It’s hard. I often answer this question by saying if it was just about building silicon it’d be super easy. It’s a 3 legged stool in servers, and servers are more complicated than the client products by orders of magnitude. It costs more money, they’re harder to do, and they give you a big challenge every day because nothing works exactly the way it should. I think it’s something that the industry hasn't really grokked because there hasn't been a lot of server suppliers.

The CPU has gotten complex in a way that we can talk about, but I think most of your listeners and viewers know the level of complexity is almost prohibitive at this point if you don't have prior CPU experience. Like maybe you’ve built 80 high-performance CPUs in a prior life, so maybe you actually have an idea at how hard this can be. Second, you have to build the whole platform. The platform part is the part that in the abstract, having a core, license it out, it just sounds so easy and great, but there’s this other thing called the platform - the packaging the IO, the power deliver, everything about that platform in server is a thing, so we built our full platform. We have packaging people, we have motherboard people, we have substrate people. You have to have a specialised substrate for these things.

And then the final frontier of happiness and joy, Software.

The software in servers is not off the shelf. You can get the base infrastructure, OS and KVM, whatever the base infrastructure you're using, but then one day you get to this thing that just doesn't do what it's supposed to, and you have to work on it. I think software optimisation, library enhancement, OS kernel work - that is a level of challenge that people who may have wanted to do this, who moved from cell phones, and then they're like ‘no, we should do servers’. I invite everyone to do this - I think everyone should be in this business, because it’s so fun.

So we’ll run into something where - I’ll give you an example. It works perfectly on this one Arm core, and then it goes to another one and then you only get 50% performance. Why is that? What’s going on? Oh, this one little thing went wrong. Now you're arms deep in Java, and the garbage collector freaking out - I mean how many people have the capability to do that? You of course, know that I used to do that for a living - as I was the head of software for a long time before I went into management. Then when I started the company, 50% of the company was software people, there’s a reason.

Ian: I hear you’re roughly 1100 people now?

Renée: We’re a little more now, but it is half software, still.

Ian: It has to be I think, in this day and age.

Renée: Yeah it does, and I do think that that’s been the stumbling block. When we started the company, as you'd expect a bunch of engineers didn't just think ‘oh yeah were definitely going to be successful’. We took a spreadsheet. We looked at all the companies that had tried this - we looked at every single one of their architectures, the choices they made. We looked at all the prior Arm attempts, and in fact we really thought Qualcomm had come very far. We asked ourselves why did they fail? Did they fail because of a structural issue in the industry? Did the fail because they didn’t know this one thing? Was it technological, was it business-oriented? I’ll just say that the spreadsheet was very informative. It proved to me that the business is more complicated than people really think.

Ian: Well I assume some of the people that built that spreadsheet were also part of some of those attempts?

Renée: They were, because they work here. I was a big fan of hiring as many of them as I could. We have an outstanding design team in Raleigh that predominantly came from Qualcomm - they are fantastic. The AMCC team had made multiple attempts and missed 2 legs of the stool, so that’s kind of a problem. But [as engineers] they are outstanding, and in fact many of our leaders are ex-AMCC. We tried to gather up as much of the community from Marvell, from everywhere, because the head of engineering and our CTO, and me, “we” were not ARM people per se. We are now, and we actually have a philosophical point of view that the ISA doesn't matter, because [holistically] the software doesn't care. But actually, it turns out the hardware does [care about ISA], and the infrastructure software cares. So we wanted all those people, and we wanted all their prior knowledge.

Ian: So from a customer base, as a cloud-native processor, obviously the focus is on the cloud service providers, you've got lots of agreements out there. You've got Azure, you've got Tencent, you’ve got Oracle, you’ve got Google. How has the journey been to build those partnerships?

Rene: The journey has been hard, and challenging. The journey is hard and challenging no matter where you work. It’s not unique to Ampere, it’s unique to the demands of that calibre of a customer, and I think from my own perspective I knew what their expectations were. So we endeavoured, as small as we were. We’re still very small, relatively. But we endeavoured to set the company up to provide the level of support that the expectation could be met from a quality, delivery, software people. It’s been hard, I’m not going to lie. I mean this is not for the faint of heart, this is rough going.

When I started the company, I said I believe this is the future of computing. I know we’ve been thinking and talking about it for a decade, that we have to deal with this issue. We knew everything is going to the cloud - we could debate this, but even proprietary old data centres are turning into cloud based technology. So we knew this was going to happen, that this was going to be viable for what we now call the enterprise. People think you just wake up one day and win these customers - that was never going to happen, and I knew it wasn't going to happen. But the most important lesson in building technology, especially when you're inventing something, is that you better have a customer target workload. If you don't have target workload, you're just building in abstract.

People always ask me about Oracle - when I was in my prior career, every time I had a new microprocessor, we had to go and sell somebody on how great it was. Guess who the hardest people in the world to sell to were?

Ian: The biggest customers? The smallest customers?

Renée: It was Oracle. Because they're so technical. Because they have their own OS. They have built their own chips. The database is so demanding - it's so specific, and they now have Java. So I had learned over the years that if you can make this new feature function work at scale there, it’s kinda a thing. So I wanted a first customer, and that was one of Andy’s things: Andy would always say ‘Don't build things in your mind, build something for someone to use, have a target, and then expand from there’.

Ian: Does it matter that some of these clients that you already have on board, or maybe are getting on board, are building their own Arm silicon?

Renée: I think they were all building their own before I even started the company.

Ian: How so?

Renée: Well, they used to all tell me at my old job they were building their own! So I’ve got to assume that’s correct.

Ian: It’s one thing to say, it’s another to build a team and do it.

Renée: But they have teams. They’ve all been producing silicon for a long time.

Ian: They seem to be doing it more today than they ever were though?

Renée: But there’s a reason.

Ian: The workload changed?

Renée: No - customers leave you, or you leave your customers.

Maybe because some people might have left their customers. You have to decide which one really happened. So when you see a weakness, and you’ve built this massive business, you're thinking how this one thing can happen.

People see CPUs like air – they’re always going to have them. It turns out they're super hard, and you might not [have them]. Then all of a sudden you start to think maybe you better take your own fate. I do think there’s been some of that in the last several years. So the questions is, is there a role for merchant market silicon long term? The answer is yes for sure - 100% of the market is not CSPs, and there is a lot of business that is merchant market. I think that we need diversity in suppliers, we need diversity in solutions, and we need to continue to pioneer and move forward. I don't think we’re going to be doing less than data processing, despite our love for GPUs. We should not probably be doing all of inference on GPUs – that’s probably not the distributed way of doing it, and it’s not the most cost effective.

So the question is if our customers do their own. Yes, they will. Will they use both of us? Probably. There are things they want to put in their silicon. Do they want to be on a merchant market timeline? No. Are they going to invent some of the stuff we’ve already invented and taped out that they don't even know about? No, they're not - because we’ve patented it.

It’s also a time thing - one of the most important things in being successful in semiconductors, not just in CPUs, is time. [You have to] be on time. That’s why when I set [Ampere] up, I said we’re going to pioneer a new development model and were going to have a new product. We’re going to be on time, and maybe it's not perfect - no one is. But then you have a little bit of gap - if you're not perfectly on time, you’ve just got to keep the pressure on for that. So that rate and pace of innovation in a BYO is hard to keep up. But these companies are massive - there's room for lots of things. My view is the best product always wins - that’s just life. Build the best product.

Ian: With the last few years, the pivot to everybody talking about machine learning, how has the shift in the dynamic of the marketplace changed how you're approaching things?

Renée: So one of the things we did 3 years ago is we acquired [a particular team of engineers]. I know - I started a start0up, I’m investor funded, and I’m like ‘Hey I want to acquire a team!’. [My investors] were all ‘have you lost your mind??’

This team, they do AI acceleration libraries. It turns out in retrospect it’s a really good idea to have, because it’s already in all our products. We acquired that team and implemented it into AmpereOne. Now we’re back to the question as to why we have our own core and microarchitecture - we do things because we have a point of view, based on a lot of experience, many things that are on accelerators can be and should be more efficiently done on a CPU. That’s not to say everything should be done on CPU, certainly not training AI, but there are a lot of things, there are a lot of other language models that actually run faster on the CPU if they're accelerated properly. We have base level acceleration - obviously I’m not going to pre-announce any additional kinds of things like that, but you can imagine that is the roadmap. I have a view- I love accelerators, and accelerators give you a vision for what people care about, but accelerators don't exist forever, unless they're very specific to a very specific workload or they do something that just can't be done for good reason. So we actually built it into our roadmap, but we didn't talk about it.

Ian: I’m looking over at Jeff (Wittich, Head of Product) in the corner over there because we’re still waiting for the architecture deep dive to see what you guys have put in it. You're hyping it up to me now, and I’m going to bother him more. His email inbox is going to be overflowing.

Renée: We are relatively secretive. I’m not going to lie - it’s not because we don't love you guys and we don't want everyone to know. We’re very proud of what we’ve done, but I’ve learned my lesson - so I’m kind of going in the Steve Jobs mode. I don't need to talk about it until it’s done, or deployed. So you mentioned some of our customers, which are AmpereOne, which are deploying. When the moment is upon us, you’ll hear us talking about it. It’s not super cool to talk about stuff before the customers have actually got their services out.

Ian: Fair enough. Given everything we’ve said, and you’ve laid out a really good proposition in the market space. There was talk a couple of years ago at what the future looks like for Ampere, and IPO was mentioned, is there anything you can say?

Renée: So we went on file, and we’re still on file, so there isn't a lot I can say. The market has not been conducive to growth companies, as you know. I had really thought the second half of 2023 would be more conducive - it was not. Interest rates have not adjusted down, although I read that maybe it would. So we need to see interest from investors in growth companies again, and semiconductor companies are different software or IP licensing companies.

I think Ampere is a very unique company - there aren't any others like us, and there haven't been anything like it in a long time. The last thing that looked like this was probably NVIDIA in the early days. Jensen gave an interview talking about how he would never do it again if he knew how hard it was going to be! It is hard, it’s that hard - there are so many barriers, how would someone do this now? How would they do this? I mean, I grew up in this, I've been in semiconductors since I got out of my undergraduate degree. I have been worrying about what has been going on with yields and wafers - I worked at an IDM, and you have to know a lot of stuff about a lot of weird things to get this right. You have to know manufacturing, you have to understand you know what's going on with the process, even if you’re not running the process. If you're us, you're using TSMC, but, it's hard.

Ian: Would you use Intel Foundry?

Renée: I would use anyone who has a leading-edge competitive process that could give us an advantage. Best. Product. Wins.

Ian: What does the future of Ampere look like? Can you lay out what the next 3-5 years looks like?

Renée: I don't think the vision is any different today than it was 6 years ago. We need to drive to be an at-scale semiconductor company. I’d like us to see the industry have more diversity of supply, and I think it’s super important that we have merchant market silicon vendors for the health of US semi. I do not think US semiconductor can be fragmented, such that everyone builds their own. I mean how does that work from the software ecosystem? That’s not a healthy software ecosystem.

We actually break the underlying premise of at-scale semis, because for every CPU there are all these other things that go around it. Think of how many suppliers. We have 50 suppliers that aren't TSMC in the supply chain to build a rack, and all of that is predicated on an environment of merchant market silicon on top of which there is an entire ecosystem of software vendors who rely on us. We can't break it, because if you break it, you break the engine of semiconductors. So I applaud the customers who have the wherewithal, the financial ability, and the skills to do it. But not everyone can do it, and I don't think that it is healthy for the future of semiconductors and innovation. For us to say “we don't need merchant market anymore” – that’s dumb. Remember why we do this.

For example, there was a point where how long did it take to synthesize human DNA? Then one day we invented something that could do it in 2 days, and it changed the world, and as a result we now have cancer centers around the world doing customised cancer treatments, because we know things we didn't know before. So I get up every day, and build these massively boring things that people don't understand, so we can find a cure to things we don't know about. That, I think, is worthy. Build-Your-Own doesn't do that - you're not trying to solve problems generically for the world, or give engines to people to solve problems. You end up solving your own problems for your own self. I don't think that’s the only thing we need to do.

Ian: Is this like asking the customer what they want, and they say a faster horse, and not vehicles?

Renée: It is exactly that. I can't believe you said that - I actually put that on a slide for my board one time! I do not want faster horses. We are actually inventors of the future of computing, that is what we’re doing here. None of us need to be doing this - we’re teaching people in an apprenticed-based business how to do this, and why you do this. The why is almost as important in this environment, because we have all these people in this business. This is I wanted to be on your show, to get in front of people who would never know me, and say ‘It matters, you should come work here, you should get here!”, you know what I mean? We can't succumb to rhetoric about BYO and bad ideas by people who aren't actually semiconductor companies.

More Than Moore, as with other research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, which may include advertising on the More Than Moore newsletter or TechTechPotato YouTube channel and related social media. The companies that fall under this banner include AMD, Applied Materials, Armari, Baidu, Facebook, IBM, Infineon, Intel, Lattice Semi, Linode, MediaTek, NordPass. NVIDIA, ProteanTecs, Qualcomm, SiFive, Supermicro, Tenstorrent, TSMC.

ncG1vNJzZmiln6eytbTAp6Sop6Kae7TBwayrmpubY7CwuY6pZpqmXZ67tbHRr6Cer12strW0jKucp52VYreiucSsZJydnw%3D%3D