Nested quotes

[nilshoerrmann]

When converting the following example using JoliTypo, nested quotes (quotes inside quotes) are not parsed and the closing quote is mistaken for an apostrophe.

Input

"This 'magic' piece of code fixes dump quotes and apostrophes."

Output

“This 'magic’ piece of code fixes dump quotes and apostrophes.” 
------^

Expected

“This ‘magic’ piece of code fixes dump quotes and apostrophes.”
------^ 

[nilshoerrmann]

As a quick fix, I duplicated the SmartQuotes fixer with rules for nested quotes. If this is run additionally, everything is converted correctly:

<?php
namespace JoliTypo\Fixer;

use JoliTypo\Exception\BadFixerConfigurationException;
use JoliTypo\Fixer;
use JoliTypo\FixerInterface;
use JoliTypo\LocaleAwareFixerInterface;
use JoliTypo\StateBag;
use JoliTypo\Fixer\BaseOpenClosePair;

class NestedQuotes extends BaseOpenClosePair implements
    FixerInterface,
    LocaleAwareFixerInterface
{
    const LSAQUO = '‹'; // &lsaquo;
    const RSAQUO = '›'; // &rsaquo;
    const LSQUO = '‘'; // &lsquo;
    const RSQUO = '’ '; // &rsquo;
    const SBQUO = '‚'; // &sdquo;

    protected $opening;
    protected $openingSuffix = '';
    protected $closing;
    protected $closingPrefix = '';

    public function __construct($locale)
    {
        $this->setLocale($locale);
    }

    public function fix($content, StateBag $stateBag = null)
    {
        if (!$this->opening || !$this->closing) {
            throw new BadFixerConfigurationException();
        }

        // Fix complex siblings cases
        if ($stateBag) {
            $content = $this->fixViaState(
                $content,
                $stateBag,
                'SmartQuotesOpenSolo',
                "@(^|\s|\()'([^']*)$@im",
                "@(^|[^']+)'@im",
                $this->opening . $this->openingSuffix,
                $this->closingPrefix . $this->closing
            );
        }

        // Fix simple cases
        $content = preg_replace(
            "@(^|\s|\()'([^']+)'@im",
            '$1' .
                $this->opening .
                $this->openingSuffix .
                '$2' .
                $this->closingPrefix .
                $this->closing,
            $content
        );

        return $content;
    }

    /**
     * Default configuration for supported lang.
     *
     * @param string $locale
     */
    public function setLocale($locale)
    {
        // Handle from locale + country
        switch (strtolower($locale)) {
            // “…”
            case 'pt-br':
                $this->opening = NestedQuotes::LSQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::RSQUO;
                $this->closingPrefix = '';

                return;
            // «…»
            case 'de-ch':
                $this->opening = NestedQuotes::LSAQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::RSAQUO;
                $this->closingPrefix = '';

                return;
        }

        // Handle from locale only
        $short = Fixer::getLanguageFromLocale($locale);

        switch ($short) {
            // « … »
            case 'fr':
                $this->opening = NestedQuotes::LSAQUO;
                $this->openingSuffix = NestedQuotes::NO_BREAK_SPACE;
                $this->closing = NestedQuotes::RSAQUO;
                $this->closingPrefix = NestedQuotes::NO_BREAK_SPACE;
                break;
            // «…»
            case 'hy':
            case 'az':
            case 'hz':
            case 'eu':
            case 'be':
            case 'ca':
            case 'el':
            case 'it':
            case 'no':
            case 'fa':
            case 'lv':
            case 'pt':
            case 'ru':
            case 'es':
            case 'uk':
                $this->opening = NestedQuotes::LSAQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::RSAQUO;
                $this->closingPrefix = '';
                break;
            // „…“
            case 'de':
            case 'ka':
            case 'cs':
            case 'et':
            case 'is':
            case 'lt':
            case 'mk':
            case 'ro':
            case 'sk':
            case 'sl':
            case 'wen':
                $this->opening = NestedQuotes::SBQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::LSQUO;
                $this->closingPrefix = '';
                break;
            // “…”
            case 'en':
            case 'us':
            case 'gb':
            case 'af':
            case 'ar':
            case 'eo':
            case 'id':
            case 'ga':
            case 'ko':
            case 'br':
            case 'th':
            case 'tr':
            case 'vi':
                $this->opening = NestedQuotes::LSQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::RSQUO;
                $this->closingPrefix = '';
                break;
            // ”…”
            case 'fi':
            case 'sv':
            case 'bs':
                $this->opening = NestedQuotes::RSQUO;
                $this->openingSuffix = '';
                $this->closing = NestedQuotes::RSQUO;
                $this->closingPrefix = '';
                break;
        }
    }

    /**
     * @param string $opening
     */
    public function setOpening($opening)
    {
        $this->opening = $opening;
    }

    /**
     * @param string $openingSuffix
     */
    public function setOpeningSuffix($openingSuffix)
    {
        $this->openingSuffix = $openingSuffix;
    }

    /**
     * @param string $closing
     */
    public function setClosing($closing)
    {
        $this->closing = $closing;
    }

    /**
     * @param string $closingPrefix
     */
    public function setClosingPrefix($closingPrefix)
    {
        $this->closingPrefix = $closingPrefix;
    }
}

Maybe this can be incorporated in the Smartquotes fixer itself.

[damienalexandre]

Thanks for reporting the issue and sharing your solution, it looks good to me - the SmartQuote fixer could do that by running first with ' and second with " I guess.

How do you feel about creation a Pull Request with this in mind, and a new test case to make sure this bug is taken care of for good?
Thanks!

[nilshoerrmann]

Yes, I'd like to do that but first need to understand the library a bit better (first time using it today). The StateBag is still a blackbox to me and the smart quote handling seems to be the most complex part to start with :)

[nilshoerrmann]

Thanks for reporting the issue and sharing your solution, it looks good to me - the SmartQuote fixer could do that by running first with ' and second with " I guess.

I noticed during testing that this cannot be merged with the smart quotes fixer because this would conflict with the curly quotes fixer. Instead it needs to be merged with the curly quotes fixer to handle edge cases as well.

[damienalexandre]

Interesting 👍

Have you tried removing the CurlyQuote fixer and using only the SmartQuote fixer?

Because the text you are fixing could match the CurlyQuote fixer.

[nilshoerrmann]

Yes, I did try that but in the end curly quotes and nested quotes have similar matching pattern so they need to be looked at simultaneously:

• If curly quote are handled first, it takes away all the closing single quotes.
• If nested quotes are handled first, it takes inner apostropes for closing quotes.

It's a question of execution order and lookahead.