In 1956, he organized the first international conference to emphasize artificial intelligence. One of the attendees was Marvin Minsky, who later became one of the main AI theorists and joined McCarthy at MIT in 1959. During the autumn of 1956, McCarthy won an MIT research fellowship. He served on the committee that designed ALGOL, which became a very influential programming language by introducing many new constructs now in common use. In 1958, he proposed the advice taker, which inspired later work on question-answering and logic programming. Around 1959, he invented so-called "garbage collection" methods to solve problems in Lisp. Based on the lambda calculus, Lisp soon became the programming language of choice for AI applications after its publication in 1960. He helped to motivate the creation of Project MAC at MIT, but left MIT for Stanford University in 1962, where he helped establish the Stanford AI Laboratory, for many years a friendly rival to Project MAC.

In 1961, he was the first to suggest publicly (in a speech given to celebrate MIT's centennial) that computer time-sharing technology might result in a future in which computing power and even specific applications could be sold through the utility business model (like water or electricity). This idea of a computer or information utility was very popular during the late 1960s, but faded by the mid-1990s. However, since 2000, the idea has resurfaced in new forms (see application service provider, grid computing, and cloud computing).

From 1978 to 1986, McCarthy developed the circumscription method of non-monotonic reasoning.McCarthy is also credited with developing an early form of time-sharing. His colleague Lester Earnest told the Los Angeles Times: "The Internet would not have happened nearly as soon as it did except for the fact that John initiated the development of time-sharing systems. We keep inventing new names for time-sharing. It came to be called servers.… Now we call it cloud computing. That is still just time-sharing. John started it."