Sam Dogen walked away from his 9-to-5 job because he was burnt out.
For 13 years, he invested at least 50% of his salary to grow his retirement portfolio.
Now, he’s able to withdraw 2% from his brokerage account and reinvest the remainder.
Sam Dogen retired from his role as an executive director at Credit Suisse in 2012 at the age of 34 after feeling burnt out from working long hours. The exhaustion wasn’t new for him. He began his career in 1999 as a financial analyst for international equities at Goldman Sachs. He’d be in the office by 5:30 a.m., and he’d work until 7:00 or 9:00 p.m. on some days. That experience made him realize he couldn’t stay in investment banking long-term, and it prompted him to save for early retirement at the start of his career, he said. Dogen told Insider that for his entire 13-year career, he invested at least 50% of his salary towards retirement regardless of how much he was making. He recalled earning $40,000 at the start of his career, not including his bonus which was $10,000 in his first year. This meant he had to lead a very frugal lifestyle, especially because he lived in New York City. He shared a studio apartment with a roommate and paid about $800 a month. His firm paid for his meals if he stayed after 7:00 p.m., something he did every night. By 2012, his salary had increased to $250,000, and he was able to sometimes invest up to 75% of it towards retirement.The catalyst for walking away that year came from being able to negotiate a severance package that paid for at least five years of living expenses, he said. “My severance package included a six-figure severance check because I had worked at my firm, Credit Suisse, for 11 years,” Dogen said. “And then it also included receiving all my deferred stock and cash compensation, which I was going to invest over a three-to-five-year period.” He told Insider his severance package was worth over $400,000 and estimated that his net worth was about $3 million when he retired in 2012.At 46, he’s still out of the workforce and is a stay-at-home father. He’s known by the online pseudonym Financial Samurai, which represents a personal finance blog and podcast. In 2022, he published the book, “Buy This, Not That: How to Spend Your Way to Wealth and Freedom,” which covers everything from paying down debt and building passive income to investing.Today, he has a portfolio of stocks and bonds that equals almost $4.4 million. That includes $2.9 million that’s in a regular brokerage account, according to statements viewed Insider. The latter portion allows him to access his retirement income before the age of 59 and a half without incurring a 10% penalty. In August, the brokerage portion of his portfolio generated $7,307 in passive income from dividends and bond interest, according to his brokerage statement. The stock exposure in that account is mainly through the iShares Core S&P 500 ETF (IVV), where he has about $1 million invested. The ETF has an effective yield of 1.48%. He has a smaller amount of about $110,000 invested in the Fidelity MSCI Information Technology Index ETF (FTEC). The remainder of his brokerage portfolio is in fixed-income securities, including long- and short-term Treasury bonds.In his first 10 years after retiring, Dogen reinvested the yields and dividends, allowing his portfolio to grow. He lived off the cash flow from rental income collected through his real-estate investments and revenue generated from his website. When he turned 45, he began withdrawing some of his yields. While most of his income still comes from real estate and his online content business, Dogen believes someone can retire early solely on a stocks-and-bonds portfolio. Still, he warns that it’s harder because you’ll need to put away more money and continually pay attention to which securities you hold and how they meet your financial needs. You also must be prepared to weather the stock market’s volatility, he added. To put it into context, the average dividend yield for stocks in the S&P 500 was 1.54% in June, according to YCharts. This roughly means you need to have $1 million invested into the index to earn $15,400. If bonds have a high yield of 5%, the same amount invested could generate $50,000 annually. However, high-yielding bonds are a recent phenomenon and may not always be around, he noted. Portfolio makeup Dogen says the first step to building the right retirement portfolio is understanding your risk tolerance and how much money you must set aside for your retirement needs. Since he has passive income from other avenues but is still withdrawing from his brokerage account, he has decided that his risk tolerance is low to medium. He wants to have exposure to stocks but wants to avoid too much volatility. This way, he doesn’t pull cash when the market is down. Therefore, bonds and a fixed-income fund account for about 58% of his brokerage portfolio. As for how he determined the amount of money to set aside, one way to go about it is by following the 4% rule, which states that you can safely withdraw 4% of your overall portfolio while allowing the remainder to continue growing. Dogen told Insider he sticks to a more conservative rate by withdrawing about 2% and reinvesting the rest. The inverse of the 4% rule is to multiply your expenses by 25 to determine what the value of your stocks-and-bonds portfolio should be before you can retire, he noted. For example, if your expenses equal $60,000 yearly, then it would be 60,000 x 25 = 1,500,000. Therefore, you would need at least $1.5 million worth of stocks and bonds. Since Dogen lives in San Francisco — a relatively expensive city — with his wife and two kids, he estimates needing at least $200,000 of income, translating to $5 million. You also need to determine the types of securities and the ratio of each that could yield the right amount of cash but match your risk tolerance, he said. Risk-free returns mainly come from Treasury bonds, good news right now because their current yields are above the historical average, he noted. As of Friday, the one-month Treasury bill’s yield was 5.38%. Dogen said he is allocating a larger amount to short-term Treasuries that are between three months to two years. “Theoretically, I should be happy to invest the majority of my portfolio in Treasury bonds because I’m a retiree, I’ve won the game,” Dogen said. “I don’t need to take more risks. I’m happy to live with what I have. So as rates go up, risk-free rates go up. The allocation towards bonds should go up and vice versa because stocks are more risky than bonds.”For more fixed-income avenues, he has a small amount, or slightly less than $10,000, invested in certificates of deposit, yielding him 4.75% — a relatively attractive rate at the time of deposit. Regarding his stock exposure, he isn’t picking and choosing different companies. Even after working in investment banking for over 13 years, he doesn’t feel savvy enough to make bets that can outperform the S&P 500 long-term. Therefore, only 1% of his brokerage portfolio is allocated to a few individual tech stocks. Since he lives in San Francisco, a tech-heavy city, most of the conversations around him are about technology companies, so he admits he has a fear of missing out on growth in that sector.The rest of his equity exposure sits in IVV. One benefit is the ETF’s low cost, with an expense ratio of 0.03%.Finally, for anyone who wants to retire early and has been at a company for at least three years, Dogen is a big proponent of negotiating a severance package if they are laid off.